2024-05-17T04:12:18.073 INFO:root:teuthology version: 0.0.1.dev275+ga4b16c5 2024-05-17T04:12:18.073 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343 --owner scheduled_pdonnell@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} --name pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/orig.config.yaml 2024-05-17T04:12:18.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T04:12:18.221 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343 branch: wip-pdonnell-testing-20240517.012458-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7710343' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.012458-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 931809a9e5d295bd61984693cac44fe131ded4bc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 931809a9e5d295bd61984693cac44fe131ded4bc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.012458-debug sha1: 931809a9e5d295bd61984693cac44fe131ded4bc owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1967 sha1: 931809a9e5d295bd61984693cac44fe131ded4bc sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.012458-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 931809a9e5d295bd61984693cac44fe131ded4bc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGBw4HRRBAuICgSAdJlNpq42ZfcfTVD3PFiXCfRSetrG/hsHKPqjjG4fekggT9FccUlhmTjXk9XFLDmCNY4qge0= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHR3UKV1SRDItX0gAumqmQCMr1Ytj23giUxBnHNgsQ9Ay370py1t196a1csrMdVp6Np67dSaAthITirqsclP7Gs= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8CPOCt/Zlul84IdYgRXZZvyZCUspTeTv47aN50ERVwK77psa1SN2D4sIWUW7YvjrKiJtHPf1VnQqUnFqSEgjM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: a4b16c58a5d7aa60dee491f6f096b7509e2f1269 timestamp: 2024-05-17_03:38:57 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-17T04:12:18.221 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa; will attempt to use it 2024-05-17T04:12:18.222 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks 2024-05-17T04:12:18.223 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-17T04:12:18.224 INFO:teuthology.task.internal:Checking packages... 2024-05-17T04:12:18.246 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '931809a9e5d295bd61984693cac44fe131ded4bc' 2024-05-17T04:12:18.246 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T04:12:18.246 INFO:teuthology.packaging:ref: None 2024-05-17T04:12:18.246 INFO:teuthology.packaging:tag: None 2024-05-17T04:12:18.246 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.012458-debug 2024-05-17T04:12:18.246 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:12:18.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.012458-debug 2024-05-17T04:12:18.462 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3794.g931809a9 2024-05-17T04:12:18.464 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-17T04:12:18.469 INFO:teuthology.task.internal:no buildpackages task found 2024-05-17T04:12:18.469 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-17T04:12:18.580 INFO:teuthology.task.internal:Saving configuration 2024-05-17T04:12:18.602 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-17T04:12:18.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T04:12:18.631 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 04:06:48.056673', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGBw4HRRBAuICgSAdJlNpq42ZfcfTVD3PFiXCfRSetrG/hsHKPqjjG4fekggT9FccUlhmTjXk9XFLDmCNY4qge0='} 2024-05-17T04:12:18.650 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 04:06:48.053858', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHR3UKV1SRDItX0gAumqmQCMr1Ytj23giUxBnHNgsQ9Ay370py1t196a1csrMdVp6Np67dSaAthITirqsclP7Gs='} 2024-05-17T04:12:18.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 04:06:48.055362', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8CPOCt/Zlul84IdYgRXZZvyZCUspTeTv47aN50ERVwK77psa1SN2D4sIWUW7YvjrKiJtHPf1VnQqUnFqSEgjM='} 2024-05-17T04:12:18.666 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-17T04:12:18.673 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-17T04:12:18.673 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-17T04:12:18.673 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-17T04:12:18.673 INFO:teuthology.run_tasks:Running task console_log... 2024-05-17T04:12:18.896 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f44b0267ac0>, signals=[15]) 2024-05-17T04:12:18.896 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-17T04:12:18.902 INFO:teuthology.task.internal:Opening connections... 2024-05-17T04:12:18.902 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:12:18.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:12:18.983 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-05-17T04:12:18.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:12:19.056 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-05-17T04:12:19.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:12:19.132 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-17T04:12:19.138 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-05-17T04:12:19.157 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-05-17T04:12:19.157 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T04:12:19.214 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T04:12:19.215 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T04:12:19.215 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-05-17T04:12:19.233 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-05-17T04:12:19.251 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-05-17T04:12:19.251 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T04:12:19.308 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T04:12:19.309 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T04:12:19.309 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-05-17T04:12:19.333 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-05-17T04:12:19.350 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-05-17T04:12:19.351 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-05-17T04:12:19.408 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-05-17T04:12:19.409 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-17T04:12:19.409 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-05-17T04:12:19.409 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-17T04:12:19.409 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-17T04:12:19.409 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-17T04:12:19.409 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-05-17T04:12:19.464 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-17T04:12:19.570 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-17T04:12:19.606 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-17T04:12:19.607 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-05-17T04:12:19.609 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-05-17T04:12:19.611 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-05-17T04:12:19.628 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-17T04:12:19.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-17T04:12:19.637 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-05-17T04:12:19.666 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-05-17T04:12:19.668 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-05-17T04:12:19.694 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-17T04:12:19.760 INFO:teuthology.run_tasks:Running task kernel... 2024-05-17T04:12:19.775 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-17T04:12:19.775 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-17T04:12:19.776 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T04:12:19.778 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T04:12:19.779 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T04:12:19.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:12:19.795 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-05-17T04:12:19.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:12:19.796 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-05-17T04:12:19.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:12:19.797 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-17T04:12:19.852 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-437.el9.x86_64 2024-05-17T04:12:19.852 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-437.el9.x86_64 2024-05-17T04:12:19.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-05-17T04:12:19.853 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-437.el9.x86_64 2024-05-17T04:12:19.854 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-437.el9.x86_64 2024-05-17T04:12:19.854 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-05-17T04:12:19.855 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-437.el9.x86_64 2024-05-17T04:12:19.855 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-437.el9.x86_64 2024-05-17T04:12:19.855 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-05-17T04:12:22.446 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.1 MB 00:00 2024-05-17T04:12:22.467 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.1 MB 00:00 2024-05-17T04:12:23.963 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.1 MB 00:02 2024-05-17T04:12:24.114 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 35 MB/s | 19 MB 00:00 2024-05-17T04:12:25.201 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-17T04:12:25.898 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-17T04:12:29.740 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-17T04:12:30.066 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-17T04:12:31.265 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-05-17T04:12:31.508 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-17T04:12:31.627 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-05-17T04:12:33.169 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-17T04:12:33.192 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-17T04:12:35.179 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-17T04:12:38.375 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-17T04:12:40.373 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T04:12:40.373 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T04:12:40.456 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-05-17T04:12:40.468 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-17T04:12:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-05-17T04:12:40.470 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:12:40.470 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 78 M 2024-05-17T04:12:40.470 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 123 M 2024-05-17T04:12:40.470 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-05-17T04:12:40.605 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-17T04:12:40.785 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 2.3 MB/s | 550 kB 00:00 2024-05-17T04:12:42.685 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T04:12:42.686 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T04:12:42.783 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-17T04:12:42.791 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-17T04:12:42.792 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-05-17T04:12:42.792 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:12:42.792 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 78 M 2024-05-17T04:12:42.792 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 123 M 2024-05-17T04:12:42.792 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-17T04:12:43.097 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 3.0 MB/s | 550 kB 00:00 2024-05-17T04:12:43.531 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 25 MB/s | 15 MB 00:00 2024-05-17T04:12:43.882 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 35 MB/s | 34 MB 00:00 2024-05-17T04:12:43.952 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 4.6 MB/s | 15 MB 00:03 2024-05-17T04:12:44.248 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 24 MB/s | 28 MB 00:01 2024-05-17T04:12:44.248 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:12:44.248 INFO:teuthology.orchestra.run.smithi050.stdout:Total 54 MB/s | 78 MB 00:01 2024-05-17T04:12:44.962 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-17T04:12:45.358 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-17T04:12:45.358 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-17T04:12:45.660 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-core-5.14.0-446.el9.x86_6 5.7 MB/s | 28 MB 00:04 2024-05-17T04:12:46.169 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 21 MB 00:14 2024-05-17T04:12:46.354 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-17T04:12:46.354 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-17T04:12:47.619 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 4.8 MB/s | 34 MB 00:07 2024-05-17T04:12:47.619 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:12:47.619 INFO:teuthology.orchestra.run.smithi105.stdout:Total 11 MB/s | 78 MB 00:07 2024-05-17T04:12:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-05-17T04:12:48.624 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-05-17T04:12:48.624 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-05-17T04:12:48.887 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-17T04:12:49.529 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:12:49.551 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-05-17T04:12:49.551 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-05-17T04:12:49.616 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:12:51.300 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-17T04:12:51.927 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-05-17T04:12:52.412 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:12:52.545 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:12:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:12:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:12:53.314 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-17T04:12:53.315 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-17T04:12:53.409 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-17T04:12:53.421 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 78 M 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 123 M 2024-05-17T04:12:53.422 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-05-17T04:12:54.485 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 556 kB/s | 550 kB 00:00 2024-05-17T04:12:55.218 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 9.0 MB/s | 15 MB 00:01 2024-05-17T04:12:55.363 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:12:55.536 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:12:55.993 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-17T04:12:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-17T04:12:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:12:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:Total 22 MB/s | 78 MB 00:03 2024-05-17T04:12:57.603 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-05-17T04:12:57.619 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:12:57.666 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:12:57.948 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-05-17T04:12:57.948 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-05-17T04:12:58.893 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-05-17T04:12:58.893 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-05-17T04:13:00.259 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:00.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:01.260 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-05-17T04:13:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:13:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:13:02.760 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:04.936 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:13:05.027 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:05.083 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:09.902 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:33.773 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:33.812 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.176 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:13:34.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:13:34.176 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:34.388 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.426 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.820 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:13:34.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:13:34.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:34.990 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T04:13:34.991 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:34.991 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:13:34.991 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-17T04:13:35.357 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:35.429 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.429 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.429 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.430 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T04:13:35.430 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T04:13:35.430 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-17T04:13:35.430 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T04:13:35.430 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:35.482 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.482 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.482 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-05-17T04:13:35.513 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-05-17T04:13:35.513 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-05-17T04:13:35.513 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-05-17T04:13:35.513 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-05-17T04:13:35.513 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-05-17T04:13:35.514 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:13:35.515 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T04:13:35.516 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T04:13:35.516 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.516 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:13:35.595 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-05-17T04:13:35.932 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:35.999 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:35.999 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T04:13:35.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T04:13:36.000 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T04:13:36.000 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-17T04:13:36.000 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T04:13:36.000 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:36.048 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:36.049 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:36.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T04:13:36.080 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:13:36.081 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T04:13:36.082 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T04:13:36.082 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-446.el9.x86_64 2024-05-17T04:13:36.082 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T04:13:36.083 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-05-17T04:13:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-05-17T04:13:37.503 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T04:13:37.539 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-05-17T04:13:37.541 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-05-17T04:13:37.557 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.Oa2T2fws2w 2024-05-17T04:13:37.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Oa2T2fws2w 2024-05-17T04:13:37.623 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.Oa2T2fws2w 2024-05-17T04:13:37.762 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.Oa2T2fws2w is 6KB 2024-05-17T04:13:37.812 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.Oa2T2fws2w 2024-05-17T04:13:37.826 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T04:13:37.899 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T04:13:37.899 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T04:13:37.899 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T04:13:37.899 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T04:13:37.900 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T04:13:38.088 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-05-17T04:13:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T04:13:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-05-17T04:13:38.119 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-05-17T04:13:38.134 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.JCSJ4VqBg1 2024-05-17T04:13:38.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JCSJ4VqBg1 2024-05-17T04:13:38.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.JCSJ4VqBg1 2024-05-17T04:13:38.340 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.JCSJ4VqBg1 is 6KB 2024-05-17T04:13:38.388 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.JCSJ4VqBg1 2024-05-17T04:13:38.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T04:13:38.470 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T04:13:38.471 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T04:13:38.471 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T04:13:38.471 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T04:13:38.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T04:13:38.619 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-05-17T04:13:43.902 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:43.940 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:44.260 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:44.260 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-17T04:13:44.260 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-17T04:13:44.260 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:13:45.131 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-05-17T04:13:45.473 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:45.553 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.554 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:45.554 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-17T04:13:45.554 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T04:13:45.554 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T04:13:45.554 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-05-17T04:13:45.554 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T04:13:45.554 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-05-17T04:13:45.604 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-17T04:13:45.604 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-17T04:13:45.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-05-17T04:13:45.640 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-05-17T04:13:45.640 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 68.el9 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8264254 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-17T04:13:45.641 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:13:45.642 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-05-17T04:13:45.643 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-17T04:13:45.643 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-446.el9.x86_64 2024-05-17T04:13:45.643 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-17T04:13:46.249 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-05-17T04:13:47.633 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-17T04:13:47.660 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-05-17T04:13:47.662 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-05-17T04:13:47.676 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.Vap68JKOLZ 2024-05-17T04:13:47.677 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Vap68JKOLZ 2024-05-17T04:13:47.742 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.Vap68JKOLZ 2024-05-17T04:13:47.880 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.Vap68JKOLZ is 6KB 2024-05-17T04:13:47.929 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.Vap68JKOLZ 2024-05-17T04:13:47.943 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-05-17T04:13:48.003 INFO:teuthology.orchestra.run.smithi110.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-17T04:13:48.003 INFO:teuthology.orchestra.run.smithi110.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-17T04:13:48.003 INFO:teuthology.orchestra.run.smithi110.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-17T04:13:48.004 INFO:teuthology.orchestra.run.smithi110.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-17T04:13:48.005 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-17T04:13:48.166 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-05-17T04:14:08.110 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-17T04:14:08.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:14:08.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:08.622 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-17T04:14:08.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:14:08.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:18.178 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-17T04:14:18.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:14:18.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:26.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-05-17T04:14:27.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-05-17T04:14:35.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:14:35.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:36.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:14:36.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:36.578 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-05-17T04:14:38.658 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-05-17T04:14:39.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-05-17T04:14:45.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:14:45.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:50.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:14:50.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:51.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:14:51.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:14:54.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-05-17T04:15:09.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:15:09.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:15:12.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-05-17T04:15:22.413 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-17T04:15:23.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:15:23.101 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-446.el9.x86_64"... 2024-05-17T04:15:23.102 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-17T04:15:23.116 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-446.el9.x86_64 2024-05-17T04:15:23.117 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T04:15:23.117 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T04:15:23.117 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T04:15:24.117 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-17T04:15:24.118 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T04:15:24.184 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2024-05-17T04:15:24.222 DEBUG:teuthology.parallel:result is None 2024-05-17T04:15:30.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:15:30.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:15:30.685 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-05-17T04:15:31.079 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:15:31.079 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-446.el9.x86_64"... 2024-05-17T04:15:31.079 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-05-17T04:15:31.096 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-446.el9.x86_64 2024-05-17T04:15:31.096 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T04:15:31.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T04:15:31.097 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T04:15:32.097 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-17T04:15:32.098 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T04:15:32.162 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-05-17T04:15:32.185 DEBUG:teuthology.parallel:result is None 2024-05-17T04:15:45.589 DEBUG:teuthology.orchestra.remote:timed out 2024-05-17T04:15:57.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:15:57.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:15:57.861 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-05-17T04:15:58.221 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:15:58.221 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-446.el9.x86_64"... 2024-05-17T04:15:58.222 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-05-17T04:15:58.237 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-446.el9.x86_64 2024-05-17T04:15:58.238 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-17T04:15:58.238 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T04:15:58.238 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-17T04:15:59.238 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-17T04:15:59.239 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T04:15:59.598 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-05-17T04:15:59.631 DEBUG:teuthology.parallel:result is None 2024-05-17T04:15:59.632 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-17T04:15:59.640 INFO:teuthology.task.internal:Creating test directory... 2024-05-17T04:15:59.640 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T04:15:59.643 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T04:15:59.646 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T04:15:59.664 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-17T04:15:59.682 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-17T04:15:59.720 INFO:teuthology.task.internal:Creating archive directory... 2024-05-17T04:15:59.720 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T04:15:59.722 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T04:15:59.725 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T04:15:59.762 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-17T04:15:59.769 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-17T04:15:59.769 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T04:15:59.795 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T04:15:59.797 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T04:15:59.829 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.830 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.836 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.841 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.841 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.847 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T04:15:59.849 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-17T04:15:59.856 INFO:teuthology.task.internal:Configuring sudo... 2024-05-17T04:15:59.856 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T04:15:59.887 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T04:15:59.892 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T04:15:59.920 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-17T04:15:59.929 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-17T04:15:59.930 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T04:15:59.957 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T04:15:59.959 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T04:15:59.977 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.109 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.238 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:16:00.238 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T04:16:00.306 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.397 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.522 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:16:00.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T04:16:00.587 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T04:16:00.682 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T04:16:00.810 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:16:00.810 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T04:16:00.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-05-17T04:16:00.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-05-17T04:16:00.882 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-05-17T04:16:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T04:16:00.919 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T04:16:00.956 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T04:16:01.360 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-17T04:16:01.369 INFO:teuthology.task.internal:Starting timer... 2024-05-17T04:16:01.369 INFO:teuthology.run_tasks:Running task pcp... 2024-05-17T04:16:01.407 INFO:teuthology.run_tasks:Running task selinux... 2024-05-17T04:16:01.439 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-17T04:16:01.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-05-17T04:16:01.502 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-05-17T04:16:01.504 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-05-17T04:16:01.568 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-05-17T04:16:01.569 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-05-17T04:16:01.636 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-05-17T04:16:01.638 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-17T04:16:01.638 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-05-17T04:16:01.669 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-05-17T04:16:01.669 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-05-17T04:16:01.710 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-05-17T04:16:01.710 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-05-17T04:16:01.793 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-05-17T04:16:01.793 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive'} 2024-05-17T04:16:01.793 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T04:16:01.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:16:01.824 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T04:16:01.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:16:01.851 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T04:16:01.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T04:16:01.882 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-17T04:16:01.883 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-05-17T04:16:01.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-05-17T04:16:01.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-05-17T04:16:01.969 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-17T04:16:01.978 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-17T04:16:01.990 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-17T04:16:01.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi105.front.sepia.ceph.com,smithi110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-17T04:20:23.717 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com')] 2024-05-17T04:20:23.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:20:23.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:20:23.809 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-17T04:20:23.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-17T04:20:23.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:20:23.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:20:23.974 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-05-17T04:20:24.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-17T04:20:24.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:20:24.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T04:20:24.135 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-05-17T04:20:24.214 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-17T04:20:24.216 INFO:teuthology.run_tasks:Running task clock... 2024-05-17T04:20:24.226 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-17T04:20:24.226 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T04:20:24.227 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T04:20:24.229 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T04:20:24.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T04:20:24.232 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T04:20:24.232 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T04:20:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T04:20:24.262 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T04:20:24.276 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T04:20:24.279 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T04:20:24.284 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-17T04:20:24.298 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-17T04:20:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-05-17T04:20:24.306 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-05-17T04:20:24.312 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-05-17T04:20:24.319 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-05-17T04:20:24.322 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-05-17T04:20:24.327 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T04:20:24.333 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-05-17T04:20:24.334 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T04:20:24.341 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T04:20:24.347 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-17T04:20:24.349 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T04:20:24.361 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-17T04:20:24.398 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-05-17T04:20:24.402 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T04:20:24.402 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-05-17T04:20:24.403 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.403 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.403 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.403 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.404 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.408 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.414 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T04:20:24.418 INFO:teuthology.run_tasks:Running task install... 2024-05-17T04:20:24.427 DEBUG:teuthology.task.install:project ceph 2024-05-17T04:20:24.428 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}} 2024-05-17T04:20:24.428 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'} 2024-05-17T04:20:24.428 INFO:teuthology.task.install:Using flavor: default 2024-05-17T04:20:24.433 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-17T04:20:24.433 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-17T04:20:24.434 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T04:20:24.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:20:24.436 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T04:20:24.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:20:24.438 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'tag': None, 'wait_for_package': False} 2024-05-17T04:20:24.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:20:24.580 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T04:20:24.580 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T04:20:24.594 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T04:20:24.594 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T04:20:24.655 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/ 2024-05-17T04:20:24.655 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3794.g931809a9 2024-05-17T04:20:24.779 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T04:20:24.779 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:20:24.779 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T04:20:24.791 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T04:20:24.791 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:20:24.791 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T04:20:24.793 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-17T04:20:24.794 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:20:24.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-17T04:20:24.804 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T04:20:24.805 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T04:20:24.819 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T04:20:24.820 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T04:20:24.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-17T04:20:24.821 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/931809a9e5d295bd61984693cac44fe131ded4bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-17T04:20:24.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T04:20:24.884 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T04:20:24.889 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-17T04:20:24.945 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T04:20:24.959 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T04:20:24.967 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-17T04:20:25.010 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-05-17T04:20:25.011 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-05-17T04:20:25.023 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-05-17T04:20:25.025 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-05-17T04:20:25.032 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-05-17T04:20:25.034 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-05-17T04:20:25.241 INFO:teuthology.orchestra.run.smithi105.stdout:57 files removed 2024-05-17T04:20:25.258 INFO:teuthology.orchestra.run.smithi110.stdout:57 files removed 2024-05-17T04:20:25.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T04:20:25.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T04:20:25.286 INFO:teuthology.orchestra.run.smithi050.stdout:57 files removed 2024-05-17T04:20:25.313 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-17T04:20:25.964 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 206 kB/s | 83 kB 00:00 2024-05-17T04:20:26.058 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-05-17T04:20:26.114 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 145 kB/s | 83 kB 00:00 2024-05-17T04:20:26.622 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-05-17T04:20:26.648 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-05-17T04:20:26.750 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-17T04:20:27.082 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-17T04:20:27.190 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-05-17T04:20:27.192 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-17T04:20:28.042 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-17T04:20:28.132 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.1 MB 00:00 2024-05-17T04:20:30.666 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-17T04:20:31.810 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-05-17T04:20:34.714 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.1 MB 00:07 2024-05-17T04:20:36.042 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-17T04:20:36.103 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-17T04:20:37.653 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-05-17T04:20:37.710 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-17T04:20:39.266 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-17T04:20:40.015 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-05-17T04:20:40.761 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 21 MB 00:02 2024-05-17T04:20:44.274 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-05-17T04:20:44.630 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-05-17T04:20:45.658 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 16 kB 00:00 2024-05-17T04:20:45.928 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-17T04:20:46.876 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-17T04:20:47.066 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:47.067 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:47.079 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T04:20:47.080 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T04:20:47.081 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T04:20:47.081 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T04:20:47.083 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T04:20:47.086 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T04:20:47.086 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T04:20:47.152 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-17T04:20:47.171 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T04:20:47.172 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T04:20:47.173 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T04:20:47.174 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T04:20:47.175 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T04:20:47.176 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T04:20:47.177 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T04:20:47.178 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T04:20:47.179 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T04:20:47.180 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T04:20:47.181 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T04:20:47.182 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T04:20:47.183 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T04:20:47.184 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout:Install 148 Packages 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 8 Packages 2024-05-17T04:20:47.185 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:20:47.186 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 219 M 2024-05-17T04:20:47.186 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-17T04:20:47.815 INFO:teuthology.orchestra.run.smithi050.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-05-17T04:20:48.263 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:48.264 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:48.275 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T04:20:48.275 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T04:20:48.276 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T04:20:48.277 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T04:20:48.278 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T04:20:48.281 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T04:20:48.281 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T04:20:48.343 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T04:20:48.362 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T04:20:48.363 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T04:20:48.364 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T04:20:48.365 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T04:20:48.366 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T04:20:48.367 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T04:20:48.368 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T04:20:48.369 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T04:20:48.370 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T04:20:48.371 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T04:20:48.372 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T04:20:48.373 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T04:20:48.374 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:Install 148 Packages 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 219 M 2024-05-17T04:20:48.375 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-05-17T04:20:48.542 INFO:teuthology.orchestra.run.smithi050.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 1.2 MB/s | 866 kB 00:00 2024-05-17T04:20:48.607 INFO:teuthology.orchestra.run.smithi050.stdout:(3/156): ceph-immutable-object-cache-19.0.0-379 2.1 MB/s | 147 kB 00:00 2024-05-17T04:20:49.157 INFO:teuthology.orchestra.run.smithi050.stdout:(4/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 3.4 MB/s | 5.4 MB 00:01 2024-05-17T04:20:49.266 INFO:teuthology.orchestra.run.smithi050.stdout:(5/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 3.4 MB/s | 2.3 MB 00:00 2024-05-17T04:20:49.288 INFO:teuthology.orchestra.run.smithi110.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-17T04:20:49.383 INFO:teuthology.orchestra.run.smithi050.stdout:(6/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-05-17T04:20:49.621 INFO:teuthology.orchestra.run.smithi110.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 2.5 MB/s | 866 kB 00:00 2024-05-17T04:20:49.714 INFO:teuthology.orchestra.run.smithi110.stdout:(3/156): ceph-immutable-object-cache-19.0.0-379 1.6 MB/s | 147 kB 00:00 2024-05-17T04:20:50.330 INFO:teuthology.orchestra.run.smithi110.stdout:(4/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 3.7 MB/s | 2.3 MB 00:00 2024-05-17T04:20:50.616 INFO:teuthology.orchestra.run.smithi050.stdout:(7/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 3.6 MB/s | 4.9 MB 00:01 2024-05-17T04:20:50.764 INFO:teuthology.orchestra.run.smithi110.stdout:(5/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-05-17T04:20:51.206 INFO:teuthology.orchestra.run.smithi110.stdout:(6/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-05-17T04:20:51.773 INFO:teuthology.orchestra.run.smithi110.stdout:(7/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 4.8 MB/s | 4.9 MB 00:01 2024-05-17T04:20:52.143 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-17T04:20:52.367 INFO:teuthology.orchestra.run.smithi050.stdout:(8/156): ceph-osd-19.0.0-3794.g931809a9.el9.x86 5.9 MB/s | 18 MB 00:02 2024-05-17T04:20:52.467 INFO:teuthology.orchestra.run.smithi050.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 252 kB/s | 25 kB 00:00 2024-05-17T04:20:53.215 INFO:teuthology.orchestra.run.smithi110.stdout:(8/156): ceph-osd-19.0.0-3794.g931809a9.el9.x86 8.8 MB/s | 18 MB 00:02 2024-05-17T04:20:53.217 INFO:teuthology.orchestra.run.smithi050.stdout:(10/156): ceph-radosgw-19.0.0-3794.g931809a9.el 3.9 MB/s | 10 MB 00:02 2024-05-17T04:20:53.273 INFO:teuthology.orchestra.run.smithi110.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 433 kB/s | 25 kB 00:00 2024-05-17T04:20:53.309 INFO:teuthology.orchestra.run.smithi050.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 2.4 MB/s | 227 kB 00:00 2024-05-17T04:20:53.385 INFO:teuthology.orchestra.run.smithi050.stdout:(12/156): cephfs-shell-19.0.0-3794.g931809a9.el 361 kB/s | 27 kB 00:00 2024-05-17T04:20:53.468 INFO:teuthology.orchestra.run.smithi050.stdout:(13/156): libcephfs-devel-19.0.0-3794.g931809a9 381 kB/s | 32 kB 00:00 2024-05-17T04:20:53.752 INFO:teuthology.orchestra.run.smithi050.stdout:(14/156): libcephfs2-19.0.0-3794.g931809a9.el9. 2.6 MB/s | 759 kB 00:00 2024-05-17T04:20:53.819 INFO:teuthology.orchestra.run.smithi050.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 2.3 MB/s | 158 kB 00:00 2024-05-17T04:20:53.848 INFO:teuthology.orchestra.run.smithi110.stdout:(10/156): ceph-radosgw-19.0.0-3794.g931809a9.el 4.9 MB/s | 10 MB 00:02 2024-05-17T04:20:53.894 INFO:teuthology.orchestra.run.smithi050.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 1.7 MB/s | 127 kB 00:00 2024-05-17T04:20:53.949 INFO:teuthology.orchestra.run.smithi110.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 2.2 MB/s | 227 kB 00:00 2024-05-17T04:20:54.041 INFO:teuthology.orchestra.run.smithi110.stdout:(12/156): cephfs-shell-19.0.0-3794.g931809a9.el 295 kB/s | 27 kB 00:00 2024-05-17T04:20:54.133 INFO:teuthology.orchestra.run.smithi110.stdout:(13/156): libcephfs-devel-19.0.0-3794.g931809a9 345 kB/s | 32 kB 00:00 2024-05-17T04:20:54.366 INFO:teuthology.orchestra.run.smithi110.stdout:(14/156): libcephfs2-19.0.0-3794.g931809a9.el9. 3.2 MB/s | 759 kB 00:00 2024-05-17T04:20:54.436 INFO:teuthology.orchestra.run.smithi050.stdout:(17/156): libradosstriper1-19.0.0-3794.g931809a 925 kB/s | 501 kB 00:00 2024-05-17T04:20:54.442 INFO:teuthology.orchestra.run.smithi110.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 2.1 MB/s | 158 kB 00:00 2024-05-17T04:20:54.453 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:54.453 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-17T04:20:54.465 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-17T04:20:54.466 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-17T04:20:54.466 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-17T04:20:54.467 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-17T04:20:54.469 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-17T04:20:54.471 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-17T04:20:54.471 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-17T04:20:54.517 INFO:teuthology.orchestra.run.smithi110.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 1.7 MB/s | 127 kB 00:00 2024-05-17T04:20:54.533 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-05-17T04:20:54.548 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-17T04:20:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-05-17T04:20:54.548 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-17T04:20:54.548 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-05-17T04:20:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-17T04:20:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-3794.g931809a9.el9 ceph 6.6 k 2024-05-17T04:20:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.4 M 2024-05-17T04:20:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 866 k 2024-05-17T04:20:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3794.g931809a9.el9 ceph 147 k 2024-05-17T04:20:54.549 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-3794.g931809a9.el9 ceph 1.5 M 2024-05-17T04:20:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 143 k 2024-05-17T04:20:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 3.5 M 2024-05-17T04:20:54.550 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 7.4 M 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 49 k 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 10 M 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-3794.g931809a9.el9 ceph 43 M 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 273 k 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 763 k 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 227 k 2024-05-17T04:20:54.551 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-3794.g931809a9.el9 ceph 27 k 2024-05-17T04:20:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 18 k 2024-05-17T04:20:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-17T04:20:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-17T04:20:54.552 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-17T04:20:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 32 k 2024-05-17T04:20:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 759 k 2024-05-17T04:20:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-3794.g931809a9.el9 ceph 127 k 2024-05-17T04:20:54.553 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-17T04:20:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-17T04:20:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-17T04:20:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-17T04:20:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-17T04:20:54.554 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-3794.g931809a9.el9 ceph 162 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-3794.g931809a9.el9 ceph 321 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 300 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-3794.g931809a9.el9 ceph 99 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 86 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.0 M 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 169 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-17T04:20:54.555 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-05-17T04:20:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-17T04:20:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-17T04:20:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-17T04:20:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-17T04:20:54.556 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.3 M 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 3.1 M 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-17T04:20:54.557 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 20 M 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 34 k 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-3794.g931809a9.el9 ceph 2.3 M 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 274 k 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-3794.g931809a9.el9 ceph 4.9 M 2024-05-17T04:20:54.558 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-3794.g931809a9.el9 ceph 18 M 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3794.g931809a9.el9 ceph-noarch 16 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-3794.g931809a9.el9 ceph 25 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-17T04:20:54.559 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-3794.g931809a9.el9 ceph 158 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-17T04:20:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 501 k 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-3794.g931809a9.el9 ceph 5.2 M 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-17T04:20:54.561 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-17T04:20:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-17T04:20:54.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-17T04:20:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-17T04:20:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-17T04:20:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-17T04:20:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-3794.g931809a9.el9 ceph 45 k 2024-05-17T04:20:54.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-3794.g931809a9.el9 ceph 136 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-17T04:20:54.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-17T04:20:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-17T04:20:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-17T04:20:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-17T04:20:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-17T04:20:54.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-17T04:20:54.567 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-17T04:20:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-17T04:20:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-17T04:20:54.568 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-17T04:20:54.569 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-17T04:20:54.570 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-17T04:20:54.571 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-17T04:20:54.572 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:Install 148 Packages 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 8 Packages 2024-05-17T04:20:54.573 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:20:54.574 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 219 M 2024-05-17T04:20:54.574 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-05-17T04:20:54.667 INFO:teuthology.orchestra.run.smithi110.stdout:(17/156): libradosstriper1-19.0.0-3794.g931809a 3.3 MB/s | 501 kB 00:00 2024-05-17T04:20:55.305 INFO:teuthology.orchestra.run.smithi105.stdout:(1/156): ceph-19.0.0-3794.g931809a9.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-17T04:20:55.901 INFO:teuthology.orchestra.run.smithi110.stdout:(18/156): librgw2-19.0.0-3794.g931809a9.el9.x86 4.2 MB/s | 5.2 MB 00:01 2024-05-17T04:20:55.993 INFO:teuthology.orchestra.run.smithi110.stdout:(19/156): python3-ceph-argparse-19.0.0-3794.g93 493 kB/s | 45 kB 00:00 2024-05-17T04:20:56.093 INFO:teuthology.orchestra.run.smithi110.stdout:(20/156): python3-ceph-common-19.0.0-3794.g9318 1.3 MB/s | 136 kB 00:00 2024-05-17T04:20:56.136 INFO:teuthology.orchestra.run.smithi105.stdout:(2/156): ceph-fuse-19.0.0-3794.g931809a9.el9.x8 1.0 MB/s | 866 kB 00:00 2024-05-17T04:20:56.185 INFO:teuthology.orchestra.run.smithi110.stdout:(21/156): python3-cephfs-19.0.0-3794.g931809a9. 1.7 MB/s | 162 kB 00:00 2024-05-17T04:20:56.337 INFO:teuthology.orchestra.run.smithi105.stdout:(3/156): ceph-immutable-object-cache-19.0.0-379 733 kB/s | 147 kB 00:00 2024-05-17T04:20:56.360 INFO:teuthology.orchestra.run.smithi110.stdout:(22/156): python3-rados-19.0.0-3794.g931809a9.e 1.8 MB/s | 321 kB 00:00 2024-05-17T04:20:56.453 INFO:teuthology.orchestra.run.smithi050.stdout:(18/156): librgw2-19.0.0-3794.g931809a9.el9.x86 2.6 MB/s | 5.2 MB 00:02 2024-05-17T04:20:56.527 INFO:teuthology.orchestra.run.smithi110.stdout:(23/156): python3-rbd-19.0.0-3794.g931809a9.el9 1.8 MB/s | 300 kB 00:00 2024-05-17T04:20:56.537 INFO:teuthology.orchestra.run.smithi050.stdout:(19/156): python3-ceph-argparse-19.0.0-3794.g93 543 kB/s | 45 kB 00:00 2024-05-17T04:20:56.611 INFO:teuthology.orchestra.run.smithi110.stdout:(24/156): python3-rgw-19.0.0-3794.g931809a9.el9 1.2 MB/s | 99 kB 00:00 2024-05-17T04:20:56.629 INFO:teuthology.orchestra.run.smithi050.stdout:(20/156): python3-ceph-common-19.0.0-3794.g9318 1.4 MB/s | 136 kB 00:00 2024-05-17T04:20:56.703 INFO:teuthology.orchestra.run.smithi110.stdout:(25/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 934 kB/s | 86 kB 00:00 2024-05-17T04:20:56.729 INFO:teuthology.orchestra.run.smithi050.stdout:(21/156): python3-cephfs-19.0.0-3794.g931809a9. 1.6 MB/s | 162 kB 00:00 2024-05-17T04:20:56.837 INFO:teuthology.orchestra.run.smithi105.stdout:(4/156): ceph-base-19.0.0-3794.g931809a9.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-05-17T04:20:56.919 INFO:teuthology.orchestra.run.smithi050.stdout:(22/156): python3-rados-19.0.0-3794.g931809a9.e 1.7 MB/s | 321 kB 00:00 2024-05-17T04:20:57.154 INFO:teuthology.orchestra.run.smithi105.stdout:(5/156): ceph-mgr-19.0.0-3794.g931809a9.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-05-17T04:20:57.423 INFO:teuthology.orchestra.run.smithi050.stdout:(23/156): python3-rbd-19.0.0-3794.g931809a9.el9 594 kB/s | 300 kB 00:00 2024-05-17T04:20:57.530 INFO:teuthology.orchestra.run.smithi050.stdout:(24/156): python3-rgw-19.0.0-3794.g931809a9.el9 938 kB/s | 99 kB 00:00 2024-05-17T04:20:57.630 INFO:teuthology.orchestra.run.smithi050.stdout:(25/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 858 kB/s | 86 kB 00:00 2024-05-17T04:20:57.654 INFO:teuthology.orchestra.run.smithi105.stdout:(6/156): ceph-mds-19.0.0-3794.g931809a9.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-17T04:20:57.695 INFO:teuthology.orchestra.run.smithi110.stdout:(26/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 3.1 MB/s | 3.0 MB 00:00 2024-05-17T04:20:58.338 INFO:teuthology.orchestra.run.smithi105.stdout:(7/156): ceph-mon-19.0.0-3794.g931809a9.el9.x86 4.1 MB/s | 4.9 MB 00:01 2024-05-17T04:20:58.362 INFO:teuthology.orchestra.run.smithi110.stdout:(27/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 253 kB/s | 169 kB 00:00 2024-05-17T04:20:58.454 INFO:teuthology.orchestra.run.smithi110.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3794.g 374 kB/s | 34 kB 00:00 2024-05-17T04:20:58.888 INFO:teuthology.orchestra.run.smithi110.stdout:(29/156): ceph-test-19.0.0-3794.g931809a9.el9.x 7.7 MB/s | 43 MB 00:05 2024-05-17T04:20:58.913 INFO:teuthology.orchestra.run.smithi110.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 312 kB/s | 143 kB 00:00 2024-05-17T04:20:59.155 INFO:teuthology.orchestra.run.smithi050.stdout:(26/156): ceph-common-19.0.0-3794.g931809a9.el9 1.7 MB/s | 20 MB 00:11 2024-05-17T04:20:59.214 INFO:teuthology.orchestra.run.smithi050.stdout:(27/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 1.9 MB/s | 3.0 MB 00:01 2024-05-17T04:20:59.306 INFO:teuthology.orchestra.run.smithi050.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3794.g 373 kB/s | 34 kB 00:00 2024-05-17T04:20:59.339 INFO:teuthology.orchestra.run.smithi050.stdout:(29/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 920 kB/s | 169 kB 00:00 2024-05-17T04:20:59.465 INFO:teuthology.orchestra.run.smithi050.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 901 kB/s | 143 kB 00:00 2024-05-17T04:20:59.938 INFO:teuthology.orchestra.run.smithi105.stdout:(8/156): ceph-radosgw-19.0.0-3794.g931809a9.el9 6.3 MB/s | 10 MB 00:01 2024-05-17T04:20:59.996 INFO:teuthology.orchestra.run.smithi110.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3794.g93180 3.2 MB/s | 3.5 MB 00:01 2024-05-17T04:21:00.021 INFO:teuthology.orchestra.run.smithi105.stdout:(9/156): ceph-selinux-19.0.0-3794.g931809a9.el9 303 kB/s | 25 kB 00:00 2024-05-17T04:21:00.155 INFO:teuthology.orchestra.run.smithi110.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3794.g93 1.7 MB/s | 274 kB 00:00 2024-05-17T04:21:00.247 INFO:teuthology.orchestra.run.smithi110.stdout:(33/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 539 kB/s | 49 kB 00:00 2024-05-17T04:21:00.322 INFO:teuthology.orchestra.run.smithi110.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3794.g9 219 kB/s | 16 kB 00:00 2024-05-17T04:21:00.422 INFO:teuthology.orchestra.run.smithi110.stdout:(35/156): ceph-volume-19.0.0-3794.g931809a9.el9 2.7 MB/s | 273 kB 00:00 2024-05-17T04:21:00.698 INFO:teuthology.orchestra.run.smithi110.stdout:(36/156): cephadm-19.0.0-3794.g931809a9.el9.noa 2.7 MB/s | 763 kB 00:00 2024-05-17T04:21:00.782 INFO:teuthology.orchestra.run.smithi110.stdout:(37/156): cephfs-top-19.0.0-3794.g931809a9.el9. 219 kB/s | 18 kB 00:00 2024-05-17T04:21:00.924 INFO:teuthology.orchestra.run.smithi110.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-05-17T04:21:01.124 INFO:teuthology.orchestra.run.smithi110.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-05-17T04:21:01.208 INFO:teuthology.orchestra.run.smithi110.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-17T04:21:01.291 INFO:teuthology.orchestra.run.smithi110.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-17T04:21:01.566 INFO:teuthology.orchestra.run.smithi110.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.5 MB/s | 1.2 MB 00:00 2024-05-17T04:21:01.667 INFO:teuthology.orchestra.run.smithi110.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-17T04:21:01.784 INFO:teuthology.orchestra.run.smithi110.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-17T04:21:01.870 INFO:teuthology.orchestra.run.smithi110.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-17T04:21:01.959 INFO:teuthology.orchestra.run.smithi110.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-17T04:21:02.031 INFO:teuthology.orchestra.run.smithi050.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3794.g93180 1.3 MB/s | 3.5 MB 00:02 2024-05-17T04:21:02.184 INFO:teuthology.orchestra.run.smithi110.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-17T04:21:02.260 INFO:teuthology.orchestra.run.smithi110.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-05-17T04:21:02.298 INFO:teuthology.orchestra.run.smithi050.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3794.g93 1.0 MB/s | 274 kB 00:00 2024-05-17T04:21:02.310 INFO:teuthology.orchestra.run.smithi110.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-17T04:21:02.360 INFO:teuthology.orchestra.run.smithi110.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-17T04:21:02.398 INFO:teuthology.orchestra.run.smithi050.stdout:(33/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 494 kB/s | 49 kB 00:00 2024-05-17T04:21:02.411 INFO:teuthology.orchestra.run.smithi110.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-17T04:21:02.482 INFO:teuthology.orchestra.run.smithi050.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3794.g9 197 kB/s | 16 kB 00:00 2024-05-17T04:21:02.503 INFO:teuthology.orchestra.run.smithi110.stdout:(52/156): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-05-17T04:21:02.553 INFO:teuthology.orchestra.run.smithi110.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-05-17T04:21:02.604 INFO:teuthology.orchestra.run.smithi110.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-17T04:21:02.647 INFO:teuthology.orchestra.run.smithi105.stdout:(10/156): ceph-common-19.0.0-3794.g931809a9.el9 2.5 MB/s | 20 MB 00:07 2024-05-17T04:21:02.654 INFO:teuthology.orchestra.run.smithi110.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-05-17T04:21:02.682 INFO:teuthology.orchestra.run.smithi050.stdout:(35/156): ceph-volume-19.0.0-3794.g931809a9.el9 1.3 MB/s | 273 kB 00:00 2024-05-17T04:21:02.704 INFO:teuthology.orchestra.run.smithi110.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T04:21:02.755 INFO:teuthology.orchestra.run.smithi110.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 910 kB/s | 45 kB 00:00 2024-05-17T04:21:02.813 INFO:teuthology.orchestra.run.smithi110.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-05-17T04:21:02.822 INFO:teuthology.orchestra.run.smithi105.stdout:(11/156): cephfs-mirror-19.0.0-3794.g931809a9.e 1.3 MB/s | 227 kB 00:00 2024-05-17T04:21:02.847 INFO:teuthology.orchestra.run.smithi110.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-05-17T04:21:02.872 INFO:teuthology.orchestra.run.smithi110.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-17T04:21:02.898 INFO:teuthology.orchestra.run.smithi110.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-17T04:21:02.923 INFO:teuthology.orchestra.run.smithi110.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-17T04:21:02.931 INFO:teuthology.orchestra.run.smithi105.stdout:(12/156): cephfs-shell-19.0.0-3794.g931809a9.el 250 kB/s | 27 kB 00:00 2024-05-17T04:21:02.948 INFO:teuthology.orchestra.run.smithi110.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-17T04:21:02.973 INFO:teuthology.orchestra.run.smithi110.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-17T04:21:02.998 INFO:teuthology.orchestra.run.smithi110.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-17T04:21:03.015 INFO:teuthology.orchestra.run.smithi105.stdout:(13/156): libcephfs-devel-19.0.0-3794.g931809a9 380 kB/s | 32 kB 00:00 2024-05-17T04:21:03.132 INFO:teuthology.orchestra.run.smithi110.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-05-17T04:21:03.175 INFO:teuthology.orchestra.run.smithi050.stdout:(36/156): ceph-test-19.0.0-3794.g931809a9.el9.x 4.0 MB/s | 43 MB 00:10 2024-05-17T04:21:03.199 INFO:teuthology.orchestra.run.smithi110.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-17T04:21:03.241 INFO:teuthology.orchestra.run.smithi110.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-17T04:21:03.250 INFO:teuthology.orchestra.run.smithi050.stdout:(37/156): cephfs-top-19.0.0-3794.g931809a9.el9. 244 kB/s | 18 kB 00:00 2024-05-17T04:21:03.275 INFO:teuthology.orchestra.run.smithi050.stdout:(38/156): cephadm-19.0.0-3794.g931809a9.el9.noa 1.3 MB/s | 763 kB 00:00 2024-05-17T04:21:03.283 INFO:teuthology.orchestra.run.smithi110.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-17T04:21:03.308 INFO:teuthology.orchestra.run.smithi110.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-17T04:21:03.323 INFO:teuthology.orchestra.run.smithi105.stdout:(14/156): libcephfs2-19.0.0-3794.g931809a9.el9. 2.4 MB/s | 759 kB 00:00 2024-05-17T04:21:03.375 INFO:teuthology.orchestra.run.smithi110.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-17T04:21:03.406 INFO:teuthology.orchestra.run.smithi105.stdout:(15/156): libcephsqlite-19.0.0-3794.g931809a9.e 1.9 MB/s | 158 kB 00:00 2024-05-17T04:21:03.426 INFO:teuthology.orchestra.run.smithi110.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-17T04:21:03.501 INFO:teuthology.orchestra.run.smithi110.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-05-17T04:21:03.515 INFO:teuthology.orchestra.run.smithi105.stdout:(16/156): librados-devel-19.0.0-3794.g931809a9. 1.1 MB/s | 127 kB 00:00 2024-05-17T04:21:03.551 INFO:teuthology.orchestra.run.smithi110.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-17T04:21:03.626 INFO:teuthology.orchestra.run.smithi110.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-05-17T04:21:03.684 INFO:teuthology.orchestra.run.smithi050.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-05-17T04:21:03.749 INFO:teuthology.orchestra.run.smithi105.stdout:(17/156): libradosstriper1-19.0.0-3794.g931809a 2.1 MB/s | 501 kB 00:00 2024-05-17T04:21:03.802 INFO:teuthology.orchestra.run.smithi110.stdout:(76/156): ceph-common-19.0.0-3794.g931809a9.el9 1.3 MB/s | 20 MB 00:14 2024-05-17T04:21:03.827 INFO:teuthology.orchestra.run.smithi110.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 128 kB/s | 26 kB 00:00 2024-05-17T04:21:03.852 INFO:teuthology.orchestra.run.smithi110.stdout:(78/156): python3-jinja2-2.11.3-5.el9.noarch.rp 824 kB/s | 248 kB 00:00 2024-05-17T04:21:03.886 INFO:teuthology.orchestra.run.smithi110.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-17T04:21:03.911 INFO:teuthology.orchestra.run.smithi110.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-05-17T04:21:03.937 INFO:teuthology.orchestra.run.smithi110.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 412 kB/s | 35 kB 00:00 2024-05-17T04:21:03.976 INFO:teuthology.orchestra.run.smithi050.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 670 kB/s | 195 kB 00:00 2024-05-17T04:21:03.996 INFO:teuthology.orchestra.run.smithi110.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-17T04:21:04.062 INFO:teuthology.orchestra.run.smithi110.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-17T04:21:04.088 INFO:teuthology.orchestra.run.smithi110.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-05-17T04:21:04.113 INFO:teuthology.orchestra.run.smithi110.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-05-17T04:21:04.118 INFO:teuthology.orchestra.run.smithi050.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 1.6 MB/s | 7.4 MB 00:04 2024-05-17T04:21:04.138 INFO:teuthology.orchestra.run.smithi110.stdout:(86/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-05-17T04:21:04.151 INFO:teuthology.orchestra.run.smithi050.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 917 kB/s | 803 kB 00:00 2024-05-17T04:21:04.155 INFO:teuthology.orchestra.run.smithi110.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-05-17T04:21:04.180 INFO:teuthology.orchestra.run.smithi110.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 269 kB 00:00 2024-05-17T04:21:04.206 INFO:teuthology.orchestra.run.smithi110.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-17T04:21:04.210 INFO:teuthology.orchestra.run.smithi050.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-17T04:21:04.223 INFO:teuthology.orchestra.run.smithi110.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-05-17T04:21:04.293 INFO:teuthology.orchestra.run.smithi050.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 750 kB/s | 106 kB 00:00 2024-05-17T04:21:04.306 INFO:teuthology.orchestra.run.smithi110.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-05-17T04:21:04.344 INFO:teuthology.orchestra.run.smithi050.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-05-17T04:21:04.382 INFO:teuthology.orchestra.run.smithi110.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-05-17T04:21:04.399 INFO:teuthology.orchestra.run.smithi110.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 479 kB/s | 44 kB 00:00 2024-05-17T04:21:04.411 INFO:teuthology.orchestra.run.smithi050.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-17T04:21:04.441 INFO:teuthology.orchestra.run.smithi110.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 859 kB/s | 36 kB 00:00 2024-05-17T04:21:04.453 INFO:teuthology.orchestra.run.smithi050.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-17T04:21:04.499 INFO:teuthology.orchestra.run.smithi110.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-05-17T04:21:04.525 INFO:teuthology.orchestra.run.smithi110.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-05-17T04:21:04.600 INFO:teuthology.orchestra.run.smithi110.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 772 kB/s | 58 kB 00:00 2024-05-17T04:21:04.628 INFO:teuthology.orchestra.run.smithi050.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-17T04:21:04.642 INFO:teuthology.orchestra.run.smithi110.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-17T04:21:04.661 INFO:teuthology.orchestra.run.smithi050.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-17T04:21:04.684 INFO:teuthology.orchestra.run.smithi110.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-17T04:21:04.687 INFO:teuthology.orchestra.run.smithi050.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-17T04:21:04.709 INFO:teuthology.orchestra.run.smithi110.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-05-17T04:21:04.712 INFO:teuthology.orchestra.run.smithi050.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-05-17T04:21:04.746 INFO:teuthology.orchestra.run.smithi050.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-17T04:21:04.759 INFO:teuthology.orchestra.run.smithi110.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-17T04:21:04.763 INFO:teuthology.orchestra.run.smithi050.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-17T04:21:04.805 INFO:teuthology.orchestra.run.smithi050.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-05-17T04:21:04.810 INFO:teuthology.orchestra.run.smithi110.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-17T04:21:04.847 INFO:teuthology.orchestra.run.smithi050.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-17T04:21:04.868 INFO:teuthology.orchestra.run.smithi110.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-05-17T04:21:04.914 INFO:teuthology.orchestra.run.smithi050.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-05-17T04:21:04.939 INFO:teuthology.orchestra.run.smithi050.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-17T04:21:04.964 INFO:teuthology.orchestra.run.smithi050.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-05-17T04:21:04.988 INFO:teuthology.orchestra.run.smithi110.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [SSL certificate problem: certificate has expired] 2024-05-17T04:21:04.990 INFO:teuthology.orchestra.run.smithi050.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-05-17T04:21:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-17T04:21:05.065 INFO:teuthology.orchestra.run.smithi050.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-05-17T04:21:05.091 INFO:teuthology.orchestra.run.smithi050.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-17T04:21:05.141 INFO:teuthology.orchestra.run.smithi050.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-17T04:21:05.166 INFO:teuthology.orchestra.run.smithi050.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-17T04:21:05.185 INFO:teuthology.orchestra.run.smithi110.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-05-17T04:21:05.188 INFO:teuthology.orchestra.run.smithi110.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [SSL certificate problem: certificate has expired] 2024-05-17T04:21:05.192 INFO:teuthology.orchestra.run.smithi050.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-17T04:21:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-17T04:21:05.341 INFO:teuthology.orchestra.run.smithi110.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [SSL certificate problem: certificate has expired] 2024-05-17T04:21:05.409 INFO:teuthology.orchestra.run.smithi050.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-17T04:21:05.451 INFO:teuthology.orchestra.run.smithi050.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-17T04:21:05.493 INFO:teuthology.orchestra.run.smithi050.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-17T04:21:05.507 INFO:teuthology.orchestra.run.smithi105.stdout:(18/156): ceph-test-19.0.0-3794.g931809a9.el9.x 7.9 MB/s | 43 MB 00:05 2024-05-17T04:21:05.543 INFO:teuthology.orchestra.run.smithi050.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-17T04:21:05.586 INFO:teuthology.orchestra.run.smithi110.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 49 kB/s | 19 kB 00:00 2024-05-17T04:21:05.599 INFO:teuthology.orchestra.run.smithi105.stdout:(19/156): librgw2-19.0.0-3794.g931809a9.el9.x86 2.8 MB/s | 5.2 MB 00:01 2024-05-17T04:21:05.610 INFO:teuthology.orchestra.run.smithi050.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-17T04:21:05.625 INFO:teuthology.orchestra.run.smithi105.stdout:(20/156): python3-ceph-argparse-19.0.0-3794.g93 387 kB/s | 45 kB 00:00 2024-05-17T04:21:05.661 INFO:teuthology.orchestra.run.smithi050.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-17T04:21:05.683 INFO:teuthology.orchestra.run.smithi105.stdout:(21/156): python3-ceph-common-19.0.0-3794.g9318 1.6 MB/s | 136 kB 00:00 2024-05-17T04:21:05.694 INFO:teuthology.orchestra.run.smithi110.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 620 kB/s | 548 kB 00:00 2024-05-17T04:21:05.708 INFO:teuthology.orchestra.run.smithi105.stdout:(22/156): python3-cephfs-19.0.0-3794.g931809a9. 1.9 MB/s | 162 kB 00:00 2024-05-17T04:21:05.753 INFO:teuthology.orchestra.run.smithi050.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-17T04:21:05.800 INFO:teuthology.orchestra.run.smithi105.stdout:(23/156): python3-rbd-19.0.0-3794.g931809a9.el9 3.2 MB/s | 300 kB 00:00 2024-05-17T04:21:05.826 INFO:teuthology.orchestra.run.smithi105.stdout:(24/156): python3-rados-19.0.0-3794.g931809a9.e 2.2 MB/s | 321 kB 00:00 2024-05-17T04:21:05.828 INFO:teuthology.orchestra.run.smithi110.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 322 kB/s | 309 kB 00:00 2024-05-17T04:21:05.851 INFO:teuthology.orchestra.run.smithi105.stdout:(25/156): python3-rgw-19.0.0-3794.g931809a9.el9 1.9 MB/s | 99 kB 00:00 2024-05-17T04:21:05.878 INFO:teuthology.orchestra.run.smithi050.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-05-17T04:21:05.903 INFO:teuthology.orchestra.run.smithi050.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-05-17T04:21:05.909 INFO:teuthology.orchestra.run.smithi105.stdout:(26/156): rbd-fuse-19.0.0-3794.g931809a9.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-17T04:21:05.928 INFO:teuthology.orchestra.run.smithi050.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-17T04:21:05.937 INFO:teuthology.orchestra.run.smithi110.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 107 kB/s | 26 kB 00:00 2024-05-17T04:21:05.954 INFO:teuthology.orchestra.run.smithi050.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-05-17T04:21:05.962 INFO:teuthology.orchestra.run.smithi110.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 363 kB/s | 49 kB 00:00 2024-05-17T04:21:05.979 INFO:teuthology.orchestra.run.smithi050.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-17T04:21:06.010 INFO:teuthology.orchestra.run.smithi105.stdout:(27/156): rbd-nbd-19.0.0-3794.g931809a9.el9.x86 1.7 MB/s | 169 kB 00:00 2024-05-17T04:21:06.013 INFO:teuthology.orchestra.run.smithi050.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-17T04:21:06.038 INFO:teuthology.orchestra.run.smithi050.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-05-17T04:21:06.046 INFO:teuthology.orchestra.run.smithi110.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-05-17T04:21:06.085 INFO:teuthology.orchestra.run.smithi105.stdout:(28/156): rbd-mirror-19.0.0-3794.g931809a9.el9. 13 MB/s | 3.0 MB 00:00 2024-05-17T04:21:06.088 INFO:teuthology.orchestra.run.smithi050.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-17T04:21:06.096 INFO:teuthology.orchestra.run.smithi110.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-17T04:21:06.110 INFO:teuthology.orchestra.run.smithi105.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3794.g 341 kB/s | 34 kB 00:00 2024-05-17T04:21:06.136 INFO:teuthology.orchestra.run.smithi105.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3794.g931809a 2.8 MB/s | 143 kB 00:00 2024-05-17T04:21:06.322 INFO:teuthology.orchestra.run.smithi050.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-17T04:21:06.347 INFO:teuthology.orchestra.run.smithi050.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 859 kB/s | 222 kB 00:00 2024-05-17T04:21:06.355 INFO:teuthology.orchestra.run.smithi110.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-05-17T04:21:06.381 INFO:teuthology.orchestra.run.smithi050.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-17T04:21:06.388 INFO:teuthology.orchestra.run.smithi110.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-17T04:21:06.406 INFO:teuthology.orchestra.run.smithi050.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-05-17T04:21:06.448 INFO:teuthology.orchestra.run.smithi050.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-17T04:21:06.455 INFO:teuthology.orchestra.run.smithi110.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-05-17T04:21:06.473 INFO:teuthology.orchestra.run.smithi050.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-17T04:21:06.489 INFO:teuthology.orchestra.run.smithi110.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-05-17T04:21:06.524 INFO:teuthology.orchestra.run.smithi050.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-17T04:21:06.539 INFO:teuthology.orchestra.run.smithi110.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-17T04:21:06.549 INFO:teuthology.orchestra.run.smithi050.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-17T04:21:06.575 INFO:teuthology.orchestra.run.smithi050.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-17T04:21:06.606 INFO:teuthology.orchestra.run.smithi110.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-05-17T04:21:06.625 INFO:teuthology.orchestra.run.smithi050.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-05-17T04:21:06.631 INFO:teuthology.orchestra.run.smithi110.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-05-17T04:21:06.657 INFO:teuthology.orchestra.run.smithi110.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-17T04:21:06.675 INFO:teuthology.orchestra.run.smithi050.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-05-17T04:21:06.690 INFO:teuthology.orchestra.run.smithi110.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-17T04:21:06.716 INFO:teuthology.orchestra.run.smithi110.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-05-17T04:21:06.725 INFO:teuthology.orchestra.run.smithi050.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-17T04:21:06.741 INFO:teuthology.orchestra.run.smithi110.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-17T04:21:06.776 INFO:teuthology.orchestra.run.smithi050.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-17T04:21:06.786 INFO:teuthology.orchestra.run.smithi105.stdout:(31/156): ceph-osd-19.0.0-3794.g931809a9.el9.x8 1.9 MB/s | 18 MB 00:09 2024-05-17T04:21:06.833 INFO:teuthology.orchestra.run.smithi110.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-17T04:21:06.835 INFO:teuthology.orchestra.run.smithi050.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-05-17T04:21:06.858 INFO:teuthology.orchestra.run.smithi110.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-05-17T04:21:06.869 INFO:teuthology.orchestra.run.smithi105.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-05-17T04:21:06.892 INFO:teuthology.orchestra.run.smithi110.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 822 kB/s | 144 kB 00:00 2024-05-17T04:21:06.911 INFO:teuthology.orchestra.run.smithi105.stdout:(33/156): ceph-mgr-rook-19.0.0-3794.g931809a9.e 1.2 MB/s | 49 kB 00:00 2024-05-17T04:21:06.917 INFO:teuthology.orchestra.run.smithi110.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-05-17T04:21:06.942 INFO:teuthology.orchestra.run.smithi110.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-17T04:21:06.945 INFO:teuthology.orchestra.run.smithi105.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3794.g9 492 kB/s | 16 kB 00:00 2024-05-17T04:21:06.967 INFO:teuthology.orchestra.run.smithi110.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-17T04:21:06.970 INFO:teuthology.orchestra.run.smithi105.stdout:(35/156): ceph-mgr-modules-core-19.0.0-3794.g93 1.5 MB/s | 274 kB 00:00 2024-05-17T04:21:06.993 INFO:teuthology.orchestra.run.smithi110.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-17T04:21:06.995 INFO:teuthology.orchestra.run.smithi105.stdout:(36/156): ceph-volume-19.0.0-3794.g931809a9.el9 5.3 MB/s | 273 kB 00:00 2024-05-17T04:21:07.010 INFO:teuthology.orchestra.run.smithi050.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-05-17T04:21:07.035 INFO:teuthology.orchestra.run.smithi110.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-05-17T04:21:07.035 INFO:teuthology.orchestra.run.smithi050.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-05-17T04:21:07.060 INFO:teuthology.orchestra.run.smithi110.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-05-17T04:21:07.060 INFO:teuthology.orchestra.run.smithi050.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-17T04:21:07.071 INFO:teuthology.orchestra.run.smithi105.stdout:(37/156): cephfs-top-19.0.0-3794.g931809a9.el9. 242 kB/s | 18 kB 00:00 2024-05-17T04:21:07.086 INFO:teuthology.orchestra.run.smithi050.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T04:21:07.102 INFO:teuthology.orchestra.run.smithi110.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-05-17T04:21:07.111 INFO:teuthology.orchestra.run.smithi050.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-17T04:21:07.144 INFO:teuthology.orchestra.run.smithi110.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-17T04:21:07.180 INFO:teuthology.orchestra.run.smithi105.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 744 kB/s | 80 kB 00:00 2024-05-17T04:21:07.186 INFO:teuthology.orchestra.run.smithi110.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-17T04:21:07.203 INFO:teuthology.orchestra.run.smithi110.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-05-17T04:21:07.211 INFO:teuthology.orchestra.run.smithi050.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-05-17T04:21:07.229 INFO:teuthology.orchestra.run.smithi110.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-17T04:21:07.262 INFO:teuthology.orchestra.run.smithi110.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-17T04:21:07.279 INFO:teuthology.orchestra.run.smithi110.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-17T04:21:07.305 INFO:teuthology.orchestra.run.smithi105.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-17T04:21:07.313 INFO:teuthology.orchestra.run.smithi110.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-17T04:21:07.330 INFO:teuthology.orchestra.run.smithi110.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 879 kB/s | 59 kB 00:00 2024-05-17T04:21:07.331 INFO:teuthology.orchestra.run.smithi105.stdout:(40/156): cephadm-19.0.0-3794.g931809a9.el9.noa 2.1 MB/s | 763 kB 00:00 2024-05-17T04:21:07.347 INFO:teuthology.orchestra.run.smithi110.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 531 kB/s | 36 kB 00:00 2024-05-17T04:21:07.372 INFO:teuthology.orchestra.run.smithi110.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 865 kB/s | 51 kB 00:00 2024-05-17T04:21:07.381 INFO:teuthology.orchestra.run.smithi105.stdout:(41/156): ceph-mgr-dashboard-19.0.0-3794.g93180 2.8 MB/s | 3.5 MB 00:01 2024-05-17T04:21:07.407 INFO:teuthology.orchestra.run.smithi105.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-17T04:21:07.414 INFO:teuthology.orchestra.run.smithi110.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-17T04:21:07.431 INFO:teuthology.orchestra.run.smithi110.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-17T04:21:07.448 INFO:teuthology.orchestra.run.smithi105.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-17T04:21:07.465 INFO:teuthology.orchestra.run.smithi110.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-17T04:21:07.490 INFO:teuthology.orchestra.run.smithi110.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-17T04:21:07.498 INFO:teuthology.orchestra.run.smithi105.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-17T04:21:07.515 INFO:teuthology.orchestra.run.smithi110.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-17T04:21:07.523 INFO:teuthology.orchestra.run.smithi105.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-17T04:21:07.549 INFO:teuthology.orchestra.run.smithi105.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-17T04:21:07.574 INFO:teuthology.orchestra.run.smithi105.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-17T04:21:07.607 INFO:teuthology.orchestra.run.smithi110.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-17T04:21:07.608 INFO:teuthology.orchestra.run.smithi105.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-05-17T04:21:07.728 INFO:teuthology.orchestra.run.smithi050.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-05-17T04:21:07.741 INFO:teuthology.orchestra.run.smithi105.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-17T04:21:07.767 INFO:teuthology.orchestra.run.smithi105.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 674 kB/s | 107 kB 00:00 2024-05-17T04:21:07.792 INFO:teuthology.orchestra.run.smithi105.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-05-17T04:21:07.826 INFO:teuthology.orchestra.run.smithi105.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-05-17T04:21:07.851 INFO:teuthology.orchestra.run.smithi105.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-05-17T04:21:07.876 INFO:teuthology.orchestra.run.smithi105.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-17T04:21:07.901 INFO:teuthology.orchestra.run.smithi105.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-17T04:21:07.927 INFO:teuthology.orchestra.run.smithi105.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-17T04:21:07.952 INFO:teuthology.orchestra.run.smithi105.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-17T04:21:07.977 INFO:teuthology.orchestra.run.smithi105.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-17T04:21:08.007 INFO:teuthology.orchestra.run.smithi110.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 5.8 MB/s | 2.3 MB 00:00 2024-05-17T04:21:08.054 INFO:teuthology.orchestra.run.smithi050.stdout:(103/156): python3-cryptography-36.0.1-4.el9.x8 319 kB/s | 1.2 MB 00:03 2024-05-17T04:21:08.066 INFO:teuthology.orchestra.run.smithi110.stdout:(150/156): librbd1-19.0.0-3794.g931809a9.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-17T04:21:08.077 INFO:teuthology.orchestra.run.smithi105.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-17T04:21:08.111 INFO:teuthology.orchestra.run.smithi105.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-05-17T04:21:08.136 INFO:teuthology.orchestra.run.smithi105.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-17T04:21:08.137 INFO:teuthology.orchestra.run.smithi050.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-17T04:21:08.162 INFO:teuthology.orchestra.run.smithi105.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-17T04:21:08.179 INFO:teuthology.orchestra.run.smithi050.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 478 kB/s | 19 kB 00:00 2024-05-17T04:21:08.187 INFO:teuthology.orchestra.run.smithi105.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-17T04:21:08.191 INFO:teuthology.orchestra.run.smithi110.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 332 kB/s | 41 kB 00:00 2024-05-17T04:21:08.212 INFO:teuthology.orchestra.run.smithi105.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-17T04:21:08.229 INFO:teuthology.orchestra.run.smithi105.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-05-17T04:21:08.250 INFO:teuthology.orchestra.run.smithi110.stdout:(152/156): NetworkManager-libnm-1.47.90-1.el9.x 7.5 MB/s | 1.8 MB 00:00 2024-05-17T04:21:08.271 INFO:teuthology.orchestra.run.smithi050.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 815 kB/s | 863 kB 00:01 2024-05-17T04:21:08.280 INFO:teuthology.orchestra.run.smithi105.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-05-17T04:21:08.355 INFO:teuthology.orchestra.run.smithi105.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-17T04:21:08.367 INFO:teuthology.orchestra.run.smithi110.stdout:(153/156): librados2-19.0.0-3794.g931809a9.el9. 3.8 MB/s | 3.3 MB 00:00 2024-05-17T04:21:08.388 INFO:teuthology.orchestra.run.smithi050.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 924 kB/s | 309 kB 00:00 2024-05-17T04:21:08.392 INFO:teuthology.orchestra.run.smithi110.stdout:(154/156): NetworkManager-tui-1.47.90-1.el9.x86 1.2 MB/s | 248 kB 00:00 2024-05-17T04:21:08.422 INFO:teuthology.orchestra.run.smithi105.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-17T04:21:08.438 INFO:teuthology.orchestra.run.smithi050.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 993 kB/s | 49 kB 00:00 2024-05-17T04:21:08.481 INFO:teuthology.orchestra.run.smithi105.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-17T04:21:08.484 INFO:teuthology.orchestra.run.smithi110.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-05-17T04:21:08.489 INFO:teuthology.orchestra.run.smithi050.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-17T04:21:08.514 INFO:teuthology.orchestra.run.smithi105.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-05-17T04:21:08.514 INFO:teuthology.orchestra.run.smithi050.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 107 kB/s | 26 kB 00:00 2024-05-17T04:21:08.564 INFO:teuthology.orchestra.run.smithi050.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-17T04:21:08.581 INFO:teuthology.orchestra.run.smithi105.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-05-17T04:21:08.656 INFO:teuthology.orchestra.run.smithi105.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-17T04:21:08.681 INFO:teuthology.orchestra.run.smithi105.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-05-17T04:21:08.731 INFO:teuthology.orchestra.run.smithi050.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-05-17T04:21:08.732 INFO:teuthology.orchestra.run.smithi105.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-17T04:21:08.765 INFO:teuthology.orchestra.run.smithi050.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-05-17T04:21:08.807 INFO:teuthology.orchestra.run.smithi050.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-05-17T04:21:08.815 INFO:teuthology.orchestra.run.smithi105.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-05-17T04:21:08.824 INFO:teuthology.orchestra.run.smithi050.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-05-17T04:21:08.840 INFO:teuthology.orchestra.run.smithi105.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-17T04:21:08.841 INFO:teuthology.orchestra.run.smithi050.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 449 kB/s | 15 kB 00:00 2024-05-17T04:21:08.866 INFO:teuthology.orchestra.run.smithi105.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-17T04:21:08.891 INFO:teuthology.orchestra.run.smithi105.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-17T04:21:08.917 INFO:teuthology.orchestra.run.smithi105.stdout:(79/156): postgresql-private-libs-13.14-1.el9.x 306 kB/s | 133 kB 00:00 2024-05-17T04:21:08.942 INFO:teuthology.orchestra.run.smithi105.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-17T04:21:08.950 INFO:teuthology.orchestra.run.smithi050.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-17T04:21:08.967 INFO:teuthology.orchestra.run.smithi105.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-17T04:21:08.991 INFO:teuthology.orchestra.run.smithi050.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-05-17T04:21:08.992 INFO:teuthology.orchestra.run.smithi105.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-17T04:21:09.009 INFO:teuthology.orchestra.run.smithi105.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-05-17T04:21:09.034 INFO:teuthology.orchestra.run.smithi110.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-05-17T04:21:09.035 INFO:teuthology.orchestra.run.smithi105.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-17T04:21:09.036 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:21:09.037 INFO:teuthology.orchestra.run.smithi110.stdout:Total 11 MB/s | 219 MB 00:20 2024-05-17T04:21:09.050 INFO:teuthology.orchestra.run.smithi050.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-17T04:21:09.060 INFO:teuthology.orchestra.run.smithi105.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-05-17T04:21:09.076 INFO:teuthology.orchestra.run.smithi050.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-05-17T04:21:09.085 INFO:teuthology.orchestra.run.smithi105.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-17T04:21:09.110 INFO:teuthology.orchestra.run.smithi105.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-17T04:21:09.126 INFO:teuthology.orchestra.run.smithi050.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-17T04:21:09.136 INFO:teuthology.orchestra.run.smithi105.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-17T04:21:09.161 INFO:teuthology.orchestra.run.smithi105.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-17T04:21:09.186 INFO:teuthology.orchestra.run.smithi105.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-17T04:21:09.193 INFO:teuthology.orchestra.run.smithi050.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-17T04:21:09.226 INFO:teuthology.orchestra.run.smithi050.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 323 kB/s | 11 kB 00:00 2024-05-17T04:21:09.268 INFO:teuthology.orchestra.run.smithi050.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-05-17T04:21:09.270 INFO:teuthology.orchestra.run.smithi105.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-17T04:21:09.295 INFO:teuthology.orchestra.run.smithi105.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-05-17T04:21:09.310 INFO:teuthology.orchestra.run.smithi050.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-05-17T04:21:09.320 INFO:teuthology.orchestra.run.smithi105.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-05-17T04:21:09.346 INFO:teuthology.orchestra.run.smithi105.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-17T04:21:09.371 INFO:teuthology.orchestra.run.smithi105.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-17T04:21:09.386 INFO:teuthology.orchestra.run.smithi050.stdout:(126/156): libarrow-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-05-17T04:21:09.397 INFO:teuthology.orchestra.run.smithi105.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 937 kB/s | 47 kB 00:00 2024-05-17T04:21:09.411 INFO:teuthology.orchestra.run.smithi050.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-17T04:21:09.422 INFO:teuthology.orchestra.run.smithi105.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-17T04:21:09.437 INFO:teuthology.orchestra.run.smithi050.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-17T04:21:09.447 INFO:teuthology.orchestra.run.smithi105.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-17T04:21:09.472 INFO:teuthology.orchestra.run.smithi105.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-17T04:21:09.487 INFO:teuthology.orchestra.run.smithi050.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-05-17T04:21:09.498 INFO:teuthology.orchestra.run.smithi105.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-17T04:21:09.523 INFO:teuthology.orchestra.run.smithi105.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-17T04:21:09.528 INFO:teuthology.orchestra.run.smithi050.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-17T04:21:09.579 INFO:teuthology.orchestra.run.smithi050.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-17T04:21:09.654 INFO:teuthology.orchestra.run.smithi050.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-05-17T04:21:09.704 INFO:teuthology.orchestra.run.smithi050.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-05-17T04:21:09.746 INFO:teuthology.orchestra.run.smithi050.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-17T04:21:09.765 INFO:teuthology.orchestra.run.smithi105.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-17T04:21:09.781 INFO:teuthology.orchestra.run.smithi050.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 846 kB/s | 28 kB 00:00 2024-05-17T04:21:09.790 INFO:teuthology.orchestra.run.smithi105.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-05-17T04:21:09.822 INFO:teuthology.orchestra.run.smithi050.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 742 kB/s | 31 kB 00:00 2024-05-17T04:21:09.931 INFO:teuthology.orchestra.run.smithi050.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-17T04:21:09.957 INFO:teuthology.orchestra.run.smithi105.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-05-17T04:21:09.973 INFO:teuthology.orchestra.run.smithi050.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-17T04:21:09.982 INFO:teuthology.orchestra.run.smithi105.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-05-17T04:21:10.008 INFO:teuthology.orchestra.run.smithi105.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-05-17T04:21:10.023 INFO:teuthology.orchestra.run.smithi050.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-05-17T04:21:10.050 INFO:teuthology.orchestra.run.smithi105.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-05-17T04:21:10.082 INFO:teuthology.orchestra.run.smithi050.stdout:(140/156): python3-typing-extensions-4.4.0-2.el 877 kB/s | 51 kB 00:00 2024-05-17T04:21:10.092 INFO:teuthology.orchestra.run.smithi105.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-17T04:21:10.159 INFO:teuthology.orchestra.run.smithi105.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-17T04:21:10.166 INFO:teuthology.orchestra.run.smithi050.stdout:(141/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-17T04:21:10.184 INFO:teuthology.orchestra.run.smithi105.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-05-17T04:21:10.190 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-05-17T04:21:10.207 INFO:teuthology.orchestra.run.smithi050.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-05-17T04:21:10.217 INFO:teuthology.orchestra.run.smithi105.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-05-17T04:21:10.260 INFO:teuthology.orchestra.run.smithi105.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-17T04:21:10.293 INFO:teuthology.orchestra.run.smithi105.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-05-17T04:21:10.308 INFO:teuthology.orchestra.run.smithi050.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-17T04:21:10.318 INFO:teuthology.orchestra.run.smithi105.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-05-17T04:21:10.337 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-05-17T04:21:10.337 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-05-17T04:21:10.344 INFO:teuthology.orchestra.run.smithi105.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-05-17T04:21:10.367 INFO:teuthology.orchestra.run.smithi050.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-17T04:21:10.369 INFO:teuthology.orchestra.run.smithi105.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-17T04:21:10.394 INFO:teuthology.orchestra.run.smithi105.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-17T04:21:10.409 INFO:teuthology.orchestra.run.smithi050.stdout:(145/156): python3-kubernetes-26.1.0-2.el9.noar 1.0 MB/s | 1.0 MB 00:00 2024-05-17T04:21:10.428 INFO:teuthology.orchestra.run.smithi105.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-05-17T04:21:10.451 INFO:teuthology.orchestra.run.smithi050.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-17T04:21:10.453 INFO:teuthology.orchestra.run.smithi105.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-05-17T04:21:10.470 INFO:teuthology.orchestra.run.smithi105.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 841 kB/s | 35 kB 00:00 2024-05-17T04:21:10.495 INFO:teuthology.orchestra.run.smithi105.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-17T04:21:10.521 INFO:teuthology.orchestra.run.smithi105.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-17T04:21:10.546 INFO:teuthology.orchestra.run.smithi105.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-17T04:21:10.572 INFO:teuthology.orchestra.run.smithi105.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-05-17T04:21:10.597 INFO:teuthology.orchestra.run.smithi105.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-05-17T04:21:10.622 INFO:teuthology.orchestra.run.smithi105.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-17T04:21:10.647 INFO:teuthology.orchestra.run.smithi105.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-17T04:21:10.673 INFO:teuthology.orchestra.run.smithi105.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-05-17T04:21:10.715 INFO:teuthology.orchestra.run.smithi105.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-17T04:21:10.740 INFO:teuthology.orchestra.run.smithi105.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-05-17T04:21:10.765 INFO:teuthology.orchestra.run.smithi105.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-17T04:21:10.790 INFO:teuthology.orchestra.run.smithi105.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-17T04:21:10.816 INFO:teuthology.orchestra.run.smithi105.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-17T04:21:10.841 INFO:teuthology.orchestra.run.smithi105.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-05-17T04:21:10.883 INFO:teuthology.orchestra.run.smithi105.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-05-17T04:21:10.908 INFO:teuthology.orchestra.run.smithi105.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-05-17T04:21:10.934 INFO:teuthology.orchestra.run.smithi105.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-05-17T04:21:10.959 INFO:teuthology.orchestra.run.smithi105.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 630 kB/s | 90 kB 00:00 2024-05-17T04:21:10.984 INFO:teuthology.orchestra.run.smithi105.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-17T04:21:11.010 INFO:teuthology.orchestra.run.smithi105.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-17T04:21:11.034 INFO:teuthology.orchestra.run.smithi050.stdout:(147/156): python3-grpcio-1.46.7-10.el9.x86_64. 1.0 MB/s | 2.0 MB 00:01 2024-05-17T04:21:11.035 INFO:teuthology.orchestra.run.smithi105.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-17T04:21:11.060 INFO:teuthology.orchestra.run.smithi105.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-17T04:21:11.085 INFO:teuthology.orchestra.run.smithi105.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-17T04:21:11.110 INFO:teuthology.orchestra.run.smithi105.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-17T04:21:11.136 INFO:teuthology.orchestra.run.smithi105.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-05-17T04:21:11.143 INFO:teuthology.orchestra.run.smithi050.stdout:(148/156): librados2-19.0.0-3794.g931809a9.el9. 4.8 MB/s | 3.3 MB 00:00 2024-05-17T04:21:11.161 INFO:teuthology.orchestra.run.smithi105.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-17T04:21:11.195 INFO:teuthology.orchestra.run.smithi105.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-17T04:21:11.245 INFO:teuthology.orchestra.run.smithi105.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-17T04:21:11.443 INFO:teuthology.orchestra.run.smithi050.stdout:(149/156): librbd1-19.0.0-3794.g931809a9.el9.x8 7.6 MB/s | 3.1 MB 00:00 2024-05-17T04:21:11.454 INFO:teuthology.orchestra.run.smithi105.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-17T04:21:11.521 INFO:teuthology.orchestra.run.smithi105.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-17T04:21:11.543 INFO:teuthology.orchestra.run.smithi050.stdout:(150/156): thrift-0.15.0-2.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-17T04:21:11.562 INFO:teuthology.orchestra.run.smithi105.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-17T04:21:11.604 INFO:teuthology.orchestra.run.smithi105.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-17T04:21:11.663 INFO:teuthology.orchestra.run.smithi105.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-05-17T04:21:11.722 INFO:teuthology.orchestra.run.smithi105.stdout:(154/156): librbd1-19.0.0-3794.g931809a9.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-05-17T04:21:11.772 INFO:teuthology.orchestra.run.smithi105.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-05-17T04:21:11.902 INFO:teuthology.orchestra.run.smithi050.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 116 kB/s | 41 kB 00:00 2024-05-17T04:21:12.114 INFO:teuthology.orchestra.run.smithi105.stdout:(156/156): librados2-19.0.0-3794.g931809a9.el9. 3.5 MB/s | 3.3 MB 00:00 2024-05-17T04:21:12.118 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:21:12.118 INFO:teuthology.orchestra.run.smithi105.stdout:Total 12 MB/s | 219 MB 00:17 2024-05-17T04:21:12.177 INFO:teuthology.orchestra.run.smithi050.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 901 kB/s | 248 kB 00:00 2024-05-17T04:21:12.311 INFO:teuthology.orchestra.run.smithi050.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 2.1 MB/s | 1.8 MB 00:00 2024-05-17T04:21:12.482 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-05-17T04:21:12.482 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-05-17T04:21:12.536 INFO:teuthology.orchestra.run.smithi050.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-17T04:21:12.578 INFO:teuthology.orchestra.run.smithi050.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-05-17T04:21:12.945 INFO:teuthology.orchestra.run.smithi050.stdout:(156/156): NetworkManager-1.47.90-1.el9.x86_64. 1.3 MB/s | 2.3 MB 00:01 2024-05-17T04:21:12.952 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:21:12.952 INFO:teuthology.orchestra.run.smithi050.stdout:Total 8.5 MB/s | 219 MB 00:25 2024-05-17T04:21:13.298 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-05-17T04:21:13.448 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-05-17T04:21:13.449 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-05-17T04:21:14.193 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-17T04:21:14.365 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-17T04:21:14.365 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-17T04:21:15.559 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-05-17T04:21:15.560 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-05-17T04:21:16.601 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-17T04:21:16.601 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-17T04:21:18.782 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-05-17T04:21:18.881 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T04:21:18.919 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T04:21:19.000 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T04:21:19.039 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T04:21:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T04:21:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T04:21:19.461 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T04:21:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:19.716 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:19.750 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:19.841 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:21:20.048 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T04:21:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T04:21:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:20.232 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:20.256 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:20.302 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:20.324 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:20.346 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:20.517 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:20.714 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:20.915 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:21.016 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:21.467 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T04:21:21.506 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T04:21:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T04:21:21.641 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T04:21:21.696 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-05-17T04:21:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T04:21:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T04:21:21.775 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T04:21:21.785 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T04:21:21.803 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T04:21:21.823 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T04:21:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T04:21:21.894 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T04:21:21.927 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T04:21:21.935 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T04:21:21.957 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T04:21:21.961 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T04:21:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T04:21:22.013 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T04:21:22.076 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T04:21:22.111 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T04:21:22.178 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T04:21:22.200 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T04:21:22.272 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T04:21:22.323 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T04:21:22.350 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T04:21:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T04:21:22.462 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:22.534 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T04:21:22.588 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:22.601 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T04:21:22.613 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:22.689 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:21:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T04:21:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T04:21:22.968 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:23.064 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:23.094 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T04:21:23.157 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:23.178 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:23.198 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:23.239 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T04:21:23.258 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-17T04:21:23.267 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T04:21:23.310 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T04:21:23.348 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T04:21:23.350 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-17T04:21:23.388 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:23.389 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-17T04:21:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T04:21:23.444 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-17T04:21:23.484 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-17T04:21:23.509 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-17T04:21:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T04:21:23.570 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:23.572 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-17T04:21:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T04:21:23.701 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T04:21:23.717 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:23.768 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T04:21:23.777 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:23.841 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T04:21:23.871 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T04:21:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T04:21:23.923 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-17T04:21:23.961 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T04:21:23.999 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T04:21:24.042 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T04:21:24.043 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:24.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T04:21:24.165 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:24.172 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:21:24.194 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:24.222 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T04:21:24.273 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T04:21:24.276 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T04:21:24.288 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:21:24.302 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T04:21:24.307 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T04:21:24.316 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:21:24.342 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T04:21:24.344 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T04:21:24.415 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T04:21:24.440 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T04:21:24.458 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T04:21:24.470 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T04:21:24.500 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-17T04:21:24.501 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T04:21:24.511 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T04:21:24.535 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T04:21:24.543 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T04:21:24.568 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T04:21:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-17T04:21:24.590 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T04:21:24.616 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:24.656 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T04:21:24.664 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T04:21:24.717 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T04:21:24.723 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:21:24.748 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T04:21:24.758 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:24.778 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T04:21:24.783 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T04:21:24.813 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 14/164 2024-05-17T04:21:24.825 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T04:21:24.835 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:24.858 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-17T04:21:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T04:21:24.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T04:21:24.894 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T04:21:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T04:21:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T04:21:24.938 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T04:21:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T04:21:24.967 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T04:21:24.996 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T04:21:25.030 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T04:21:25.037 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T04:21:25.038 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T04:21:25.089 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T04:21:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T04:21:25.148 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T04:21:25.164 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T04:21:25.207 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-17T04:21:25.236 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T04:21:25.279 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T04:21:25.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T04:21:25.310 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T04:21:25.350 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T04:21:25.357 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:25.357 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T04:21:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-17T04:21:25.456 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T04:21:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T04:21:25.514 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:25.594 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:25.816 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-17T04:21:25.879 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T04:21:25.931 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-17T04:21:25.962 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T04:21:25.965 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-17T04:21:25.991 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T04:21:26.000 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-17T04:21:26.034 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T04:21:26.072 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T04:21:26.077 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-17T04:21:26.115 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T04:21:26.116 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-17T04:21:26.168 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-17T04:21:26.209 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-17T04:21:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T04:21:26.245 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-17T04:21:26.260 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T04:21:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T04:21:26.317 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T04:21:26.317 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-17T04:21:26.325 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T04:21:26.377 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-17T04:21:26.406 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-17T04:21:26.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T04:21:26.427 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T04:21:26.449 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-17T04:21:26.472 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T04:21:26.492 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T04:21:26.529 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-17T04:21:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T04:21:26.568 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 32/164 2024-05-17T04:21:26.576 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T04:21:26.580 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T04:21:26.612 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 33/164 2024-05-17T04:21:26.613 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T04:21:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T04:21:26.640 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 34/164 2024-05-17T04:21:26.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T04:21:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T04:21:26.717 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-17T04:21:26.748 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T04:21:26.764 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-5.el9.noarch 36/164 2024-05-17T04:21:26.789 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T04:21:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T04:21:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T04:21:26.843 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-17T04:21:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T04:21:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T04:21:26.991 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-17T04:21:27.052 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-17T04:21:27.147 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T04:21:27.184 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T04:21:27.226 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T04:21:27.323 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T04:21:27.359 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T04:21:27.400 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T04:21:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T04:21:27.465 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T04:21:27.550 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T04:21:27.558 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T04:21:27.634 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-17T04:21:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T04:21:27.683 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T04:21:27.716 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-17T04:21:27.799 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-17T04:21:27.800 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T04:21:27.800 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T04:21:27.800 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T04:21:27.801 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-17T04:21:27.832 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T04:21:27.839 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 44/164 2024-05-17T04:21:27.864 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T04:21:27.882 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-17T04:21:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T04:21:27.928 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T04:21:27.961 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T04:21:27.995 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T04:21:28.029 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T04:21:28.060 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-17T04:21:28.118 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T04:21:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-17T04:21:28.182 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T04:21:28.216 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T04:21:28.216 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-17T04:21:28.263 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T04:21:28.277 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-17T04:21:28.351 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-17T04:21:28.361 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T04:21:28.386 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-17T04:21:28.409 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T04:21:28.423 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-17T04:21:28.472 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-17T04:21:28.506 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-17T04:21:28.550 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-17T04:21:28.592 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-17T04:21:28.737 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-17T04:21:28.780 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-17T04:21:28.810 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-17T04:21:28.852 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-17T04:21:28.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T04:21:28.953 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-17T04:21:28.985 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-17T04:21:29.020 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T04:21:29.026 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-17T04:21:29.048 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T04:21:29.059 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-17T04:21:29.081 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T04:21:29.083 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-17T04:21:29.140 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T04:21:29.171 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-17T04:21:29.191 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T04:21:29.225 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T04:21:29.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-17T04:21:29.326 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T04:21:29.350 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-17T04:21:29.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T04:21:29.378 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-17T04:21:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T04:21:29.412 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-17T04:21:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:29.451 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-17T04:21:29.463 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T04:21:29.491 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-17T04:21:29.502 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T04:21:29.518 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:29.521 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-17T04:21:29.549 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T04:21:29.555 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-17T04:21:29.589 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-17T04:21:29.590 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T04:21:29.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-17T04:21:29.665 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-17T04:21:29.679 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T04:21:29.716 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T04:21:29.755 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-17T04:21:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T04:21:29.813 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-17T04:21:29.843 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-17T04:21:29.891 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-17T04:21:30.002 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-17T04:21:30.054 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-17T04:21:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T04:21:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:30.194 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:30.224 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T04:21:30.458 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T04:21:30.581 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T04:21:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-17T04:21:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-17T04:21:30.926 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-17T04:21:31.033 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-17T04:21:31.085 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-17T04:21:31.177 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-17T04:21:31.220 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-17T04:21:31.405 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-17T04:21:31.434 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-17T04:21:31.516 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-17T04:21:31.863 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T04:21:31.937 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T04:21:31.965 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T04:21:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T04:21:32.218 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-17T04:21:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T04:21:32.346 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-17T04:21:32.369 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:32.460 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:32.488 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T04:21:32.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T04:21:32.733 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T04:21:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:33.135 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:33.165 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T04:21:33.303 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:33.340 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:33.436 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:33.436 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T04:21:33.436 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T04:21:33.437 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:33.466 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:33.704 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-17T04:21:33.805 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-17T04:21:33.833 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-17T04:21:33.866 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-17T04:21:34.201 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-17T04:21:34.246 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:34.337 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 101/164 2024-05-17T04:21:34.367 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 102/164 2024-05-17T04:21:34.409 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-17T04:21:34.928 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-17T04:21:34.957 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:35.047 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 105/164 2024-05-17T04:21:35.075 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 106/164 2024-05-17T04:21:35.400 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:35.437 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:35.996 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:36.040 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:36.135 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:36.135 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T04:21:36.135 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T04:21:36.135 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:36.166 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:37.352 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:37.390 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:37.981 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 107/164 2024-05-17T04:21:38.019 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 108/164 2024-05-17T04:21:38.106 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-17T04:21:38.106 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-17T04:21:38.106 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:38.137 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-05-17T04:21:45.048 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-05-17T04:21:45.049 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-05-17T04:21:45.049 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:45.235 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:45.236 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T04:21:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:46.841 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:46.868 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:46.869 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:46.869 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T04:21:46.869 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:46.869 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:46.869 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:47.091 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:47.115 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-05-17T04:21:47.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:47.575 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:47.599 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:47.706 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T04:21:47.755 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:47.763 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T04:21:47.774 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:47.774 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:47.775 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T04:21:47.775 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:47.818 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T04:21:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T04:21:48.051 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T04:21:48.130 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T04:21:48.218 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T04:21:48.359 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T04:21:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T04:21:48.442 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T04:21:48.469 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T04:21:48.930 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:48.959 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T04:21:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T04:21:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:49.359 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:49.386 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:49.609 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:50.093 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:50.117 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:50.117 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:50.117 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T04:21:50.118 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:50.118 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:50.118 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 109/164 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-05-17T04:21:50.263 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-05-17T04:21:50.264 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-05-17T04:21:50.264 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-05-17T04:21:50.264 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:50.271 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T04:21:50.327 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T04:21:50.452 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:50.469 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 110/164 2024-05-17T04:21:50.470 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:50.470 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-17T04:21:50.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:50.513 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T04:21:50.577 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T04:21:50.646 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T04:21:50.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T04:21:50.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T04:21:50.920 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T04:21:50.951 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T04:21:50.979 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T04:21:51.140 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:51.159 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:51.270 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:51.425 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:51.432 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T04:21:51.452 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:51.461 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:51.486 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T04:21:51.537 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:51.546 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T04:21:51.549 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:51.584 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:51.646 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T04:21:51.707 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T04:21:52.122 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:52.150 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 111/164 2024-05-17T04:21:52.150 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.150 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-17T04:21:52.151 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:52.151 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-17T04:21:52.151 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:52.389 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 112/164 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-17T04:21:52.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T04:21:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T04:21:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:52.779 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.779 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T04:21:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:52.824 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:52.847 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.847 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T04:21:52.847 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:52.848 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:52.907 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 113/164 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-17T04:21:52.932 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:53.047 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-17T04:21:53.105 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-17T04:21:53.158 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:53.169 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-17T04:21:53.181 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:53.181 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:53.181 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T04:21:53.181 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:53.182 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:53.182 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:21:53.366 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-17T04:21:53.417 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-17T04:21:53.499 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-17T04:21:53.597 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-17T04:21:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:53.669 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:53.741 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-17T04:21:53.782 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:53.784 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-17T04:21:53.816 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-17T04:21:53.842 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-17T04:21:53.941 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T04:21:53.971 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:53.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:54.039 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:54.148 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T04:21:54.198 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T04:21:54.282 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:54.308 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 125/164 2024-05-17T04:21:54.341 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 126/164 2024-05-17T04:21:54.405 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 127/164 2024-05-17T04:21:54.431 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:55.155 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T04:21:55.191 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:55.215 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:55.215 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:55.215 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T04:21:55.215 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:55.215 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:55.216 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:55.258 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:55.282 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:55.282 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:55.282 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T04:21:55.282 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:55.334 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:55.358 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:55.535 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 128/164 2024-05-17T04:21:55.578 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:55.668 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:55.692 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:21:56.592 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 129/164 2024-05-17T04:21:56.617 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:56.736 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 130/164 2024-05-17T04:21:56.905 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 131/164 2024-05-17T04:21:56.935 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:56.961 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 132/164 2024-05-17T04:21:56.961 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:56.961 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-17T04:21:56.961 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:56.962 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-17T04:21:56.962 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:57.011 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:57.023 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 133/164 2024-05-17T04:21:57.126 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 134/164 2024-05-17T04:21:57.182 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 135/164 2024-05-17T04:21:57.429 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T04:21:57.470 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T04:21:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T04:21:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T04:21:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T04:21:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T04:21:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T04:21:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T04:21:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T04:21:57.910 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T04:21:58.176 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-17T04:21:58.214 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:58.239 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 137/164 2024-05-17T04:21:58.239 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:58.239 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-17T04:21:58.239 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:58.239 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-17T04:21:58.240 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:58.281 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:58.306 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3794.g93180 138/164 2024-05-17T04:21:58.306 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:58.306 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-17T04:21:58.306 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:58.357 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:58.382 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 139/164 2024-05-17T04:21:58.383 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:58.383 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-17T04:21:58.383 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:58.383 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-17T04:21:58.383 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:21:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:21:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:21:58.714 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:58.738 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 140/164 2024-05-17T04:21:58.738 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-17T04:21:58.739 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-17T04:21:58.739 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:58.739 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-17T04:21:58.739 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:21:59.384 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T04:21:59.472 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T04:21:59.519 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T04:21:59.606 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T04:21:59.642 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T04:21:59.668 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T04:21:59.668 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:21:59.736 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:21:59.769 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T04:21:59.769 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:21:59.853 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:21:59.878 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T04:21:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:21:59.885 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:21:59.993 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T04:22:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T04:22:00.072 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:00.072 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:00.079 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T04:22:00.111 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T04:22:00.139 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T04:22:00.157 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:00.231 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:00.325 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T04:22:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T04:22:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T04:22:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T04:22:00.492 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T04:22:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:01.181 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:01.201 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:02.031 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T04:22:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T04:22:02.162 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T04:22:02.258 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T04:22:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T04:22:02.315 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T04:22:02.316 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:22:02.391 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:22:02.424 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T04:22:02.424 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:22:02.509 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:22:02.534 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T04:22:02.541 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:02.541 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:02.719 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:02.720 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:02.804 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:02.868 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:03.292 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 141/164 2024-05-17T04:22:03.338 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 142/164 2024-05-17T04:22:03.379 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 143/164 2024-05-17T04:22:03.409 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-17T04:22:03.437 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-17T04:22:03.643 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 146/164 2024-05-17T04:22:03.692 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-17T04:22:03.722 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 148/164 2024-05-17T04:22:03.752 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-17T04:22:03.797 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-17T04:22:04.379 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:04.420 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:04.442 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-17T04:22:05.348 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 152/164 2024-05-17T04:22:05.433 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-17T04:22:05.481 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-17T04:22:05.576 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-17T04:22:05.612 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-17T04:22:05.638 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-17T04:22:05.638 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:22:05.705 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-17T04:22:05.747 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-17T04:22:05.747 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:22:05.840 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-17T04:22:05.864 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-17T04:22:05.871 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:05.872 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T04:22:05.979 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T04:22:05.980 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T04:22:05.981 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T04:22:05.982 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T04:22:05.982 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T04:22:05.982 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T04:22:06.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T04:22:06.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T04:22:06.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T04:22:06.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T04:22:06.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T04:22:06.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T04:22:06.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T04:22:06.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T04:22:06.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T04:22:06.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T04:22:06.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T04:22:06.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T04:22:06.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T04:22:06.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T04:22:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T04:22:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T04:22:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T04:22:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T04:22:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T04:22:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T04:22:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T04:22:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T04:22:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T04:22:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T04:22:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T04:22:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T04:22:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T04:22:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T04:22:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T04:22:06.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T04:22:06.014 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T04:22:06.014 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T04:22:06.014 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T04:22:06.014 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T04:22:06.014 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T04:22:06.015 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T04:22:06.016 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T04:22:06.017 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T04:22:06.018 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T04:22:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T04:22:06.020 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T04:22:06.020 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T04:22:06.021 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T04:22:06.021 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T04:22:06.021 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T04:22:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T04:22:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T04:22:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T04:22:06.066 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-17T04:22:06.067 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:06.151 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-17T04:22:06.228 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:22:08.659 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T04:22:08.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T04:22:08.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T04:22:08.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T04:22:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T04:22:08.664 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T04:22:08.665 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T04:22:08.666 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T04:22:08.667 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T04:22:08.668 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T04:22:08.669 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T04:22:08.670 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T04:22:08.671 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T04:22:08.671 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T04:22:08.671 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T04:22:08.671 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T04:22:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T04:22:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T04:22:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T04:22:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T04:22:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T04:22:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T04:22:08.807 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T04:22:08.808 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:22:08.809 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-05-17T04:22:09.236 DEBUG:teuthology.parallel:result is None 2024-05-17T04:22:11.376 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:11.376 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:22:11.376 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-05-17T04:22:11.376 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T04:22:11.377 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.378 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:11.379 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.380 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T04:22:11.381 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T04:22:11.382 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T04:22:11.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T04:22:11.384 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T04:22:11.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T04:22:11.387 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T04:22:11.388 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T04:22:11.389 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T04:22:11.390 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T04:22:11.390 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-17T04:22:11.390 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-05-17T04:22:11.808 DEBUG:teuthology.parallel:result is None 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-3794.g931809a9.el9.x86_64 1/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 3/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 4/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3794.g93180 5/164 2024-05-17T04:22:11.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 6/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 7/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 8/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 9/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 10/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 11/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 12/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 13/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 14/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_ 15/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 16/164 2024-05-17T04:22:11.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 17/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-3794.g931809a9.el9.x86_6 18/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86 19/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 20/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3794.g931809a9.el 21/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-3794.g931809a9.el9. 22/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_6 23/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 24/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 25/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 26/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 27/164 2024-05-17T04:22:11.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 28/164 2024-05-17T04:22:11.983 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 29/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3794.g931809a9. 30/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noa 31/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.n 32/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3794.g931 33/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el 34/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 35/164 2024-05-17T04:22:11.999 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.e 36/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 37/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-3794.g931809a9.el9.noarch 38/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 39/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-17T04:22:12.000 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-17T04:22:12.001 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-17T04:22:12.002 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-17T04:22:12.003 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-17T04:22:12.004 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-17T04:22:12.005 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-17T04:22:12.006 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 112/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-17T04:22:12.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-17T04:22:12.008 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-17T04:22:12.009 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-3794.g931809a9.el9.x86_64 149/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-17T04:22:12.010 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 151/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-17T04:22:12.011 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-17T04:22:12.012 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-17T04:22:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-17T04:22:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:22:14.497 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-05-17T04:22:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-17T04:22:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-3794.g931809a9.el9.noarch 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-17T04:22:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-17T04:22:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-17T04:22:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-17T04:22:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-17T04:22:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-17T04:22:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-17T04:22:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-17T04:22:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-3794.g931809a9.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-17T04:22:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-17T04:22:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-17T04:22:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:22:14.511 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-17T04:22:14.966 DEBUG:teuthology.parallel:result is None 2024-05-17T04:22:14.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:15.210 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T04:22:15.235 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.235 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.235 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T04:22:15.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:15.380 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T04:22:15.408 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.408 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.408 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T04:22:15.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:15.634 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-17T04:22:15.660 INFO:teuthology.orchestra.run.smithi110.stdout:19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.660 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794.g931809a9.el9 2024-05-17T04:22:15.660 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794.g931809a9 is installed. 2024-05-17T04:22:15.662 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-17T04:22:15.663 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:22:15.663 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T04:22:15.695 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:22:15.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T04:22:15.728 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:22:15.728 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T04:22:15.760 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-17T04:22:15.761 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:22:15.761 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T04:22:15.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T04:22:15.858 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:22:15.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T04:22:15.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T04:22:15.957 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:22:15.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T04:22:15.988 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T04:22:16.056 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-17T04:22:16.056 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:22:16.057 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T04:22:16.087 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T04:22:16.157 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:22:16.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T04:22:16.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T04:22:16.257 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:22:16.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T04:22:16.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T04:22:16.355 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-17T04:22:16.355 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:22:16.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T04:22:16.386 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T04:22:16.456 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:22:16.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T04:22:16.486 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T04:22:16.555 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:22:16.555 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T04:22:16.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T04:22:16.654 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-17T04:22:16.757 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-17T04:22:16.757 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:16.757 INFO:tasks.cephadm:Cluster fsid is 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:22:16.757 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-17T04:22:16.758 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.50', 'mon.b': '172.21.15.105', 'mon.c': '172.21.15.110'} 2024-05-17T04:22:16.758 INFO:tasks.cephadm:First mon is mon.a on smithi050 2024-05-17T04:22:16.758 INFO:tasks.cephadm:First mgr is x 2024-05-17T04:22:16.758 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-17T04:22:16.758 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-05-17T04:22:16.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-05-17T04:22:16.819 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-05-17T04:22:16.848 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-17T04:22:16.848 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:17.083 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79481/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3794-g931809a9', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3794.g931809a9'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 02:49:31.325180', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.012458-debug', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79479/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3794-g931809a9', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3794.g931809a9'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 02:47:11.778341', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.012458-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-17T04:22:17.300 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-pdonnell-testing-20240517.012458-debug, sha1 931809a9e5d295bd61984693cac44fe131ded4bc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:22:17.301 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm 2024-05-17T04:22:17.301 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm 2024-05-17T04:22:17.301 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:17.861 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 04:22 /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:17.861 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.256 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 04:22 /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.256 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.813 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 782889 May 17 04:22 /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.813 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.834 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.852 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-17T04:22:18.887 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc on all hosts... 2024-05-17T04:22:18.887 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T04:22:18.890 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T04:22:18.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc pull 2024-05-17T04:22:19.785 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T04:22:19.830 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T04:22:19.936 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T04:22:53.471 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:22:53.472 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:23:07.663 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-17T04:23:07.663 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T04:23:07.663 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T04:23:07.664 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-05-17T04:23:07.664 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T04:23:07.664 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-05-17T04:23:07.664 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-17T04:23:09.170 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-05-17T04:23:09.170 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T04:23:09.171 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c", 2024-05-17T04:23:09.171 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-05-17T04:23:09.171 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b" 2024-05-17T04:23:09.171 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-05-17T04:23:09.171 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-05-17T04:23:09.192 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-05-17T04:23:09.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-05-17T04:23:09.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-05-17T04:23:09.286 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-05-17T04:23:09.318 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-05-17T04:23:09.343 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-05-17T04:23:09.373 INFO:tasks.cephadm:Writing seed config... 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-17T04:23:09.378 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-17T04:23:09.379 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-17T04:23:09.380 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-17T04:23:09.381 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-17T04:23:09.382 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:23:09.382 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-17T04:23:09.401 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0b3ae782-1405-11ef-bc9a-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-17T04:23:09.402 DEBUG:teuthology.orchestra.run.smithi050:mon.a> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service 2024-05-17T04:23:09.444 DEBUG:teuthology.orchestra.run.smithi050:mgr.x> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-17T04:23:09.487 INFO:tasks.cephadm:Bootstrapping... 2024-05-17T04:23:09.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc -v bootstrap --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.50 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:23:09.693 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-17T04:23:09.694 INFO:teuthology.orchestra.run.smithi050.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc', '-v', 'bootstrap', '--fsid', '0b3ae782-1405-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.50', '--skip-admin-label'] 2024-05-17T04:23:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-17T04:23:09.694 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying podman|docker is present... 2024-05-17T04:23:09.720 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 5.0.2 2024-05-17T04:23:09.720 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying lvm2 is present... 2024-05-17T04:23:09.720 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying time synchronization is in place... 2024-05-17T04:23:09.728 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T04:23:09.728 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T04:23:09.734 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T04:23:09.735 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:23:09.741 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-05-17T04:23:09.748 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-05-17T04:23:09.748 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-05-17T04:23:09.748 INFO:teuthology.orchestra.run.smithi050.stdout:Repeating the final host check... 2024-05-17T04:23:09.773 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 5.0.2 2024-05-17T04:23:09.773 INFO:teuthology.orchestra.run.smithi050.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-17T04:23:09.773 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl is present 2024-05-17T04:23:09.773 INFO:teuthology.orchestra.run.smithi050.stdout:lvcreate is present 2024-05-17T04:23:09.780 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-17T04:23:09.780 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-17T04:23:09.786 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-17T04:23:09.786 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:23:09.793 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout enabled 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout active 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:Unit chronyd.service is enabled and running 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:Host looks OK 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:Cluster fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:Acquiring lock 140421138091552 on /run/cephadm/0b3ae782-1405-11ef-bc9a-c7b262605968.lock 2024-05-17T04:23:09.800 INFO:teuthology.orchestra.run.smithi050.stdout:Lock 140421138091552 acquired on /run/cephadm/0b3ae782-1405-11ef-bc9a-c7b262605968.lock 2024-05-17T04:23:09.801 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 3300 ... 2024-05-17T04:23:09.801 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying IP 172.21.15.50 port 6789 ... 2024-05-17T04:23:09.801 INFO:teuthology.orchestra.run.smithi050.stdout:Base mon IP(s) is [172.21.15.50:3300, 172.21.15.50:6789], mon addrv is [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-05-17T04:23:09.804 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.50 metric 100 2024-05-17T04:23:09.805 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.50 metric 100 2024-05-17T04:23:09.807 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-17T04:23:09.807 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-17T04:23:09.807 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-05-17T04:23:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-17T04:23:09.811 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-05-17T04:23:09.811 INFO:teuthology.orchestra.run.smithi050.stdout:Mon IP `172.21.15.50` is in CIDR network `172.21.0.0/20` 2024-05-17T04:23:09.811 INFO:teuthology.orchestra.run.smithi050.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-17T04:23:09.811 INFO:teuthology.orchestra.run.smithi050.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-17T04:23:09.812 INFO:teuthology.orchestra.run.smithi050.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T04:23:12.881 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stdout 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc... 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Getting image source signatures 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:03bb380d3011cf6565f6cdcba1828bdc1bc4b91a44dc7ba4b31647b2a6675cd0 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Copying config sha256:8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c 2024-05-17T04:23:12.882 INFO:teuthology.orchestra.run.smithi050.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-17T04:23:13.836 INFO:teuthology.orchestra.run.smithi050.stdout:ceph: stdout ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev) 2024-05-17T04:23:13.836 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph version: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev) 2024-05-17T04:23:13.836 INFO:teuthology.orchestra.run.smithi050.stdout:Extracting ceph user uid/gid from container image... 2024-05-17T04:23:14.882 INFO:teuthology.orchestra.run.smithi050.stdout:stat: stdout 167 167 2024-05-17T04:23:14.882 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial keys... 2024-05-17T04:23:15.830 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQCz20ZmSoawDRAAztqxzrfUQNqAgMw1a42yYg== 2024-05-17T04:23:16.712 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQC020ZmIvvJCRAAomTyviW5NOFa5oXYXyxGjA== 2024-05-17T04:23:17.575 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph-authtool: stdout AQC120Zm+OMfAxAAGd0hVVXKExDO/pDJwO3+Og== 2024-05-17T04:23:17.576 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial monmap... 2024-05-17T04:23:20.807 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T04:23:20.807 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-17T04:23:20.807 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:20.807 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T04:23:20.807 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool for a [v2:172.21.15.50:3300,v1:172.21.15.50:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-17T04:23:20.808 INFO:teuthology.orchestra.run.smithi050.stdout:setting min_mon_release = quincy 2024-05-17T04:23:20.808 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: set fsid to 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:20.808 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-17T04:23:20.808 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:23:20.808 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mon... 2024-05-17T04:23:21.896 INFO:teuthology.orchestra.run.smithi050.stdout:create mon.a on 2024-05-17T04:23:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-17T04:23:22.254 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-17T04:23:22.440 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target. 2024-05-17T04:23:22.440 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target. 2024-05-17T04:23:22.689 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a 2024-05-17T04:23:22.689 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-17T04:23:22.860 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target.wants/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@.service. 2024-05-17T04:23:23.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:23 smithi050 podman[31352]: 2024-05-17 04:23:23.317750459 +0000 UTC m=+0.326199697 container start e62bd500f1960dccb78b9579725c7d435ead151b093cd2ac2f3d974fad0a51a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-17T04:23:23.428 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T04:23:23.428 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-17T04:23:23.435 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T04:23:23.436 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:23:23.436 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-17T04:23:23.436 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T04:23:23.436 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon to start... 2024-05-17T04:23:23.436 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mon... 2024-05-17T04:23:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:23 smithi050 ceph-mon[31370]: mkfs 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:23 smithi050 ceph-mon[31370]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T04:23:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:23 smithi050 bash[31352]: e62bd500f1960dccb78b9579725c7d435ead151b093cd2ac2f3d974fad0a51a0 2024-05-17T04:23:23.673 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:23 smithi050 systemd[1]: Started Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: monmap epoch 1 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: last_changed 2024-05-17T04:23:17.907940+0000 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: min_mon_release 19 (squid) 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: election_strategy: 1 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:23:24.541 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: fsmap 2024-05-17T04:23:24.542 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T04:23:24.542 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: mgrmap e1: no daemons active 2024-05-17T04:23:24.542 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:24 smithi050 ceph-mon[31370]: from='client.? 172.21.15.50:0/1994500010' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-17T04:23:24.627 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout cluster: 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout id: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout services: 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.621142s) 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout data: 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-17T04:23:24.628 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout pgs: 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.905+0000 7f8c7fb22700 1 Processor -- start 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.905+0000 7f8c7fb22700 1 -- start start 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7fb22700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7fb22700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7807ce20 con 0x7f8c78109e00 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7d8be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7d8be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50550/0 (socket says 172.21.15.50:50550) 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7d8be700 1 -- 172.21.15.50:0/389558065 learned_addr learned my addr 172.21.15.50:0/389558065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.906+0000 7f8c7d8be700 1 -- 172.21.15.50:0/389558065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7810c760 con 0x7f8c78109e00 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7d8be700 1 --2- 172.21.15.50:0/389558065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c740098a0 tx=0x7f8c74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3608e0f82d1706 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7c8bc700 1 -- 172.21.15.50:0/389558065 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c74004160 con 0x7f8c78109e00 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7c8bc700 1 -- 172.21.15.50:0/389558065 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74004300 con 0x7f8c78109e00 2024-05-17T04:23:24.629 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7c8bc700 1 -- 172.21.15.50:0/389558065 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c740036a0 con 0x7f8c78109e00 2024-05-17T04:23:24.630 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7fb22700 1 -- 172.21.15.50:0/389558065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 msgr2=0x7f8c7810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:24.630 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.907+0000 7f8c7fb22700 1 --2- 172.21.15.50:0/389558065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8c740098a0 tx=0x7f8c74009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:24.630 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 -- 172.21.15.50:0/389558065 shutdown_connections 2024-05-17T04:23:24.630 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 --2- 172.21.15.50:0/389558065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7810c220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:24.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 -- 172.21.15.50:0/389558065 >> 172.21.15.50:0/389558065 conn(0x7f8c78103af0 msgr2=0x7f8c78105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:24.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 -- 172.21.15.50:0/389558065 shutdown_connections 2024-05-17T04:23:24.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 -- 172.21.15.50:0/389558065 wait complete. 2024-05-17T04:23:24.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 Processor -- start 2024-05-17T04:23:24.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.908+0000 7f8c7fb22700 1 -- start start 2024-05-17T04:23:24.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7fb22700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7fb22700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7810cbd0 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7d8be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7d8be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50562/0 (socket says 172.21.15.50:50562) 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7d8be700 1 -- 172.21.15.50:0/1994500010 learned_addr learned my addr 172.21.15.50:0/1994500010 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7d8be700 1 -- 172.21.15.50:0/1994500010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819d3f0 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7d8be700 1 --2- 172.21.15.50:0/1994500010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c74003db0 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74011630 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7819d680 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.909+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c74011950 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.910+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a0290 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.910+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c74011af0 con 0x7f8c78109e00 2024-05-17T04:23:24.639 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.910+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c7401c450 con 0x7f8c78109e00 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.910+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7810a290 con 0x7f8c78109e00 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.913+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8c74032020 con 0x7f8c78109e00 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.989+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8c78064510 con 0x7f8c78109e00 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.989+0000 7f8c6effd700 1 -- 172.21.15.50:0/1994500010 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f8c74004260 con 0x7f8c78109e00 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 msgr2=0x7f8c7819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 --2- 172.21.15.50:0/1994500010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c74004750 comp rx=0 tx=0).stop 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 shutdown_connections 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 --2- 172.21.15.50:0/1994500010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c78109e00 0x7f8c7819ceb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 >> 172.21.15.50:0/1994500010 conn(0x7f8c78103af0 msgr2=0x7f8c78105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 shutdown_connections 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:23.990+0000 7f8c7fb22700 1 -- 172.21.15.50:0/1994500010 wait complete. 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:mon is available 2024-05-17T04:23:24.640 INFO:teuthology.orchestra.run.smithi050.stdout:Assimilating anything we can from ceph.conf... 2024-05-17T04:23:25.460 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:25 smithi050 ceph-mon[31370]: from='client.? 172.21.15.50:0/1276199342' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T04:23:25.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:25 smithi050 ceph-mon[31370]: from='client.? 172.21.15.50:0/1276199342' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-17T04:23:25.729 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:25.729 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T04:23:25.729 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:25.729 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T04:23:25.729 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.50:3300,v1:172.21.15.50:6789] 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T04:23:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9f277700 1 Processor -- start 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9f277700 1 -- start start 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9f277700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9f277700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98108c10 con 0x7f2e98108230 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9d013700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9d013700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50578/0 (socket says 172.21.15.50:50578) 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.063+0000 7f2e9d013700 1 -- 172.21.15.50:0/3902319066 learned_addr learned my addr 172.21.15.50:0/3902319066 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.064+0000 7f2e9d013700 1 -- 172.21.15.50:0/3902319066 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98109400 con 0x7f2e98108230 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.064+0000 7f2e9d013700 1 --2- 172.21.15.50:0/3902319066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098a0 tx=0x7f2e94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5fb1bcc99fe920 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:25.731 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.064+0000 7f2e8ffff700 1 -- 172.21.15.50:0/3902319066 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e94004160 con 0x7f2e98108230 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.064+0000 7f2e8ffff700 1 -- 172.21.15.50:0/3902319066 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94004300 con 0x7f2e98108230 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.064+0000 7f2e8ffff700 1 -- 172.21.15.50:0/3902319066 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e940036a0 con 0x7f2e98108230 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 -- 172.21.15.50:0/3902319066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 msgr2=0x7f2e98108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 --2- 172.21.15.50:0/3902319066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098a0 tx=0x7f2e94009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 -- 172.21.15.50:0/3902319066 shutdown_connections 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 --2- 172.21.15.50:0/3902319066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e98108640 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 -- 172.21.15.50:0/3902319066 >> 172.21.15.50:0/3902319066 conn(0x7f2e98103a40 msgr2=0x7f2e98105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 -- 172.21.15.50:0/3902319066 shutdown_connections 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.065+0000 7f2e9f277700 1 -- 172.21.15.50:0/3902319066 wait complete. 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9f277700 1 Processor -- start 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9f277700 1 -- start start 2024-05-17T04:23:25.732 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9f277700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9f277700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98109930 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9d013700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9d013700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50584/0 (socket says 172.21.15.50:50584) 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9d013700 1 -- 172.21.15.50:0/1276199342 learned_addr learned my addr 172.21.15.50:0/1276199342 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9d013700 1 -- 172.21.15.50:0/1276199342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9819d3d0 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.066+0000 7f2e9d013700 1 --2- 172.21.15.50:0/1276199342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000c00 tx=0x7f2e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e94003db0 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94011630 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9819d660 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e94011950 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981a0270 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e94011af0 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.067+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e9401c450 con 0x7f2e98108230 2024-05-17T04:23:25.733 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.068+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e98109ff0 con 0x7f2e98108230 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.070+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2e94032020 con 0x7f2e98108230 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.140+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2e98064510 con 0x7f2e98108230 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.144+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f2e94004260 con 0x7f2e98108230 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.145+0000 7f2e8e7fc700 1 -- 172.21.15.50:0/1276199342 <== mon.0 v2:172.21.15.50:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2e9401c650 con 0x7f2e98108230 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.145+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 msgr2=0x7f2e9819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.145+0000 7f2e9f277700 1 --2- 172.21.15.50:0/1276199342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000c00 tx=0x7f2e94004750 comp rx=0 tx=0).stop 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.146+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 shutdown_connections 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.146+0000 7f2e9f277700 1 --2- 172.21.15.50:0/1276199342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e98108230 0x7f2e9819ce90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.146+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 >> 172.21.15.50:0/1276199342 conn(0x7f2e98103a40 msgr2=0x7f2e981935c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.146+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 shutdown_connections 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:25.146+0000 7f2e9f277700 1 -- 172.21.15.50:0/1276199342 wait complete. 2024-05-17T04:23:25.734 INFO:teuthology.orchestra.run.smithi050.stdout:Generating new minimal ceph.conf... 2024-05-17T04:23:26.912 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:26 smithi050 ceph-mon[31370]: from='client.? 172.21.15.50:0/711467251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.543+0000 7f065e558700 1 Processor -- start 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f065e558700 1 -- start start 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f065e558700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f065e558700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0658108c10 con 0x7f0658108230 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f0657fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f0657fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50594/0 (socket says 172.21.15.50:50594) 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f0657fff700 1 -- 172.21.15.50:0/2600865126 learned_addr learned my addr 172.21.15.50:0/2600865126 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.544+0000 7f0657fff700 1 -- 172.21.15.50:0/2600865126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0658109400 con 0x7f0658108230 2024-05-17T04:23:27.110 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f0657fff700 1 --2- 172.21.15.50:0/2600865126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f0648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc0b8bf64412d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f06577fe700 1 -- 172.21.15.50:0/2600865126 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0648004160 con 0x7f0658108230 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f06577fe700 1 -- 172.21.15.50:0/2600865126 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0648004300 con 0x7f0658108230 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f06577fe700 1 -- 172.21.15.50:0/2600865126 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06480119e0 con 0x7f0658108230 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f065e558700 1 -- 172.21.15.50:0/2600865126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 msgr2=0x7f0658108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.545+0000 7f065e558700 1 --2- 172.21.15.50:0/2600865126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f0648009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 -- 172.21.15.50:0/2600865126 shutdown_connections 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 --2- 172.21.15.50:0/2600865126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f0658108640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 -- 172.21.15.50:0/2600865126 >> 172.21.15.50:0/2600865126 conn(0x7f0658103a40 msgr2=0x7f0658105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 -- 172.21.15.50:0/2600865126 shutdown_connections 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 -- 172.21.15.50:0/2600865126 wait complete. 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 Processor -- start 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.546+0000 7f065e558700 1 -- start start 2024-05-17T04:23:27.111 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f065e558700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f065e558700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f065810c7b0 con 0x7f0658108230 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0657fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0657fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50606/0 (socket says 172.21.15.50:50606) 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0657fff700 1 -- 172.21.15.50:0/711467251 learned_addr learned my addr 172.21.15.50:0/711467251 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0657fff700 1 -- 172.21.15.50:0/711467251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065819db90 con 0x7f0658108230 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0657fff700 1 --2- 172.21.15.50:0/711467251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0648003f20 tx=0x7f0648011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06480038b0 con 0x7f0658108230 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0648003a50 con 0x7f0658108230 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065819de20 con 0x7f0658108230 2024-05-17T04:23:27.112 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.547+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0648019550 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.548+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581a0a30 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.548+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0648019b90 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.548+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0648024730 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.549+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0658108640 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.551+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0648043ec0 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.622+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0658064550 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.622+0000 7f0655ffb700 1 -- 172.21.15.50:0/711467251 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0648032020 con 0x7f0658108230 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 msgr2=0x7f065819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 --2- 172.21.15.50:0/711467251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0648003f20 tx=0x7f0648011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 shutdown_connections 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 --2- 172.21.15.50:0/711467251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0658108230 0x7f065819d650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 >> 172.21.15.50:0/711467251 conn(0x7f0658103a40 msgr2=0x7f0658193ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:27.113 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 shutdown_connections 2024-05-17T04:23:27.114 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:26.623+0000 7f065e558700 1 -- 172.21.15.50:0/711467251 wait complete. 2024-05-17T04:23:27.114 INFO:teuthology.orchestra.run.smithi050.stdout:Restarting the monitor... 2024-05-17T04:23:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 systemd[1]: Stopping Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:23:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a[31366]: 2024-05-17T04:23:27.362+0000 7f5498ac0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:23:27.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a[31366]: 2024-05-17T04:23:27.362+0000 7f5498ac0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-17T04:23:27.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 podman[31615]: 2024-05-17 04:23:27.598832674 +0000 UTC m=+0.322951338 container died e62bd500f1960dccb78b9579725c7d435ead151b093cd2ac2f3d974fad0a51a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T04:23:28.198 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 podman[31615]: 2024-05-17 04:23:27.939517817 +0000 UTC m=+0.663636480 container cleanup e62bd500f1960dccb78b9579725c7d435ead151b093cd2ac2f3d974fad0a51a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-17T04:23:28.198 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:27 smithi050 bash[31615]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a 2024-05-17T04:23:28.198 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 podman[31627]: 2024-05-17 04:23:28.099232201 +0000 UTC m=+0.497163405 container remove e62bd500f1960dccb78b9579725c7d435ead151b093cd2ac2f3d974fad0a51a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:23:28.525 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-17T04:23:28.525 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 systemd[1]: Stopped Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:23:28.525 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 systemd[1]: Starting Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:23:28.776 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 podman[31720]: 2024-05-17 04:23:28.525349343 +0000 UTC m=+0.097038975 container create ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-17T04:23:28.776 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 podman[31720]: 2024-05-17 04:23:28.445708892 +0000 UTC m=+0.017398482 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T04:23:28.776 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 podman[31720]: 2024-05-17 04:23:28.728013441 +0000 UTC m=+0.299703032 container init ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:23:28.776 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 podman[31720]: 2024-05-17 04:23:28.731155295 +0000 UTC m=+0.302844886 container start ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: pidfile_write: ignore empty --pid-file 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: load: jerasure load: lrc 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: RocksDB version: 7.9.2 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Git sha 0 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: DB SUMMARY 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: DB Session ID: ACU3T3Z916QXHXPR824R 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: CURRENT file: CURRENT 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-17T04:23:28.777 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104076 ; 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.error_if_exists: 0 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.create_if_missing: 0 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.paranoid_checks: 1 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.env: 0x5604ee2d8500 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.info_log: 0x5604efae0080 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.statistics: (nil) 2024-05-17T04:23:28.778 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.use_fsync: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_log_file_size: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_fallocate: 1 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.use_direct_reads: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.db_log_dir: 2024-05-17T04:23:28.779 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.wal_dir: 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.write_buffer_manager: 0x5604ef994b40 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T04:23:28.780 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.unordered_write: 0 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.row_cache: None 2024-05-17T04:23:28.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.wal_filter: None 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.two_write_queues: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.wal_compression: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.atomic_flush: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.log_readahead_size: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T04:23:28.783 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_background_jobs: 2 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_background_compactions: -1 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_subcompactions: 1 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T04:23:28.784 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_open_files: -1 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_background_flushes: -1 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Compression algorithms supported: 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kZSTD supported: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kXpressCompression supported: 0 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kZlibCompression supported: 1 2024-05-17T04:23:28.785 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kSnappyCompression supported: 1 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kLZ4Compression supported: 1 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: kBZip2Compression supported: 0 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.merge_operator: 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_filter: None 2024-05-17T04:23:28.786 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5604efae0240) 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_type: 0 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_index_type: 0 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_shortening: 1 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: checksum: 4 2024-05-17T04:23:28.787 INFO:journalctl@ceph.mon.a.smithi050.stdout: no_block_cache: 0 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache: 0x5604ef9dd1f0 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_options: 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: capacity : 536870912 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_shard_bits : 4 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: strict_capacity_limit : 0 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_cache_compressed: (nil) 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: persistent_cache: (nil) 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size: 4096 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_size_deviation: 10 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_restart_interval: 16 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: index_block_restart_interval: 1 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: metadata_block_size: 4096 2024-05-17T04:23:28.788 INFO:journalctl@ceph.mon.a.smithi050.stdout: partition_filters: 0 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: use_delta_encoding: 1 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: filter_policy: bloomfilter 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: whole_key_filtering: 1 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: verify_compression: 0 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: format_version: 5 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_index_compression: 1 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: block_align: 0 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: max_auto_readahead_size: 262144 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: prepopulate_block_cache: 0 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: initial_auto_readahead_size: 8192 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T04:23:28.789 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression: NoCompression 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.num_levels: 7 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T04:23:28.790 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T04:23:28.791 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T04:23:28.792 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T04:23:28.792 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T04:23:28.792 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T04:23:28.792 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T04:23:28.792 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T04:23:28.793 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T04:23:28.794 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T04:23:28.795 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.inplace_update_support: 0 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.bloom_locality: 0 2024-05-17T04:23:28.796 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.max_successive_merges: 0 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.ttl: 2592000 2024-05-17T04:23:28.797 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enable_blob_files: false 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.min_blob_size: 0 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T04:23:28.798 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cd3c49e5-441c-4b93-8d5c-0f9870da17ef 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919808771886, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919808772980, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97250, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13915, "raw_average_key_size": 49, "raw_value_size": 89646, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715919808, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cd3c49e5-441c-4b93-8d5c-0f9870da17ef", "db_session_id": "ACU3T3Z916QXHXPR824R", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919808773068, "job": 1, "event": "recovery_finished"} 2024-05-17T04:23:28.799 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5604ef9bfc00 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: DB pointer 0x5604f47b4000 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** DB Stats ** 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T04:23:28.800 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: L0 2/0 98.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Sum 2/0 98.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** Compaction Stats [default] ** 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.801 INFO:journalctl@ceph.mon.a.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Cumulative compaction: 0.00 GB write, 27.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Interval compaction: 0.00 GB write, 27.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache BinnedLRUCache@0x5604ef9dd1f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T04:23:28.802 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: starting mon.a rank 0 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???) e1 preinit fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).mds e1 new map 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).mds e1 print_map 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: e1 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: legacy client fscid: -1 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout: No filesystems configured 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:23:28.803 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:23:28.804 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-17T04:23:28.812 INFO:teuthology.orchestra.run.smithi050.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 bash[31720]: ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 systemd[1]: Started Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: monmap epoch 1 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: last_changed 2024-05-17T04:23:17.907940+0000 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: min_mon_release 19 (squid) 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: election_strategy: 1 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:23:29.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: fsmap 2024-05-17T04:23:29.078 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: osdmap e1: 0 total, 0 up, 0 in 2024-05-17T04:23:29.078 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:28 smithi050 ceph-mon[31739]: mgrmap e1: no daemons active 2024-05-17T04:23:29.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f2930b0c700 1 Processor -- start 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f2930b0c700 1 -- start start 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f2930b0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f2930b0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f292c07d7a0 con 0x7f292c109e50 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f292a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f292a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44878/0 (socket says 172.21.15.50:44878) 2024-05-17T04:23:29.925 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f292a59c700 1 -- 172.21.15.50:0/3031178452 learned_addr learned my addr 172.21.15.50:0/3031178452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.298+0000 7f292a59c700 1 -- 172.21.15.50:0/3031178452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c10c7b0 con 0x7f292c109e50 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f292a59c700 1 --2- 172.21.15.50:0/3031178452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f291c0098a0 tx=0x7f291c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300439180a7414a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f292959a700 1 -- 172.21.15.50:0/3031178452 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c004160 con 0x7f292c109e50 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f292959a700 1 -- 172.21.15.50:0/3031178452 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f291c004300 con 0x7f292c109e50 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f292959a700 1 -- 172.21.15.50:0/3031178452 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c0119e0 con 0x7f292c109e50 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f2930b0c700 1 -- 172.21.15.50:0/3031178452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 msgr2=0x7f292c10c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.299+0000 7f2930b0c700 1 --2- 172.21.15.50:0/3031178452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f291c0098a0 tx=0x7f291c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 -- 172.21.15.50:0/3031178452 shutdown_connections 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 --2- 172.21.15.50:0/3031178452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c10c270 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 -- 172.21.15.50:0/3031178452 >> 172.21.15.50:0/3031178452 conn(0x7f292c103a40 msgr2=0x7f292c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 -- 172.21.15.50:0/3031178452 shutdown_connections 2024-05-17T04:23:29.926 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 -- 172.21.15.50:0/3031178452 wait complete. 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.300+0000 7f2930b0c700 1 Processor -- start 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f2930b0c700 1 -- start start 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f2930b0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f2930b0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f292c10d6b0 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f292a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f292a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44890/0 (socket says 172.21.15.50:44890) 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f292a59c700 1 -- 172.21.15.50:0/2835688018 learned_addr learned my addr 172.21.15.50:0/2835688018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f292a59c700 1 -- 172.21.15.50:0/2835688018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c19dbb0 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f292a59c700 1 --2- 172.21.15.50:0/2835688018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f291c003f20 tx=0x7f291c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c0038b0 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.301+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c19de40 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.302+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f291c003a50 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.302+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c019550 con 0x7f292c109e50 2024-05-17T04:23:29.927 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.302+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c1a0a50 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.302+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f291c019be0 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.302+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f291c024730 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.303+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c10a260 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.305+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f291c043ec0 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.375+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f292c1a1040 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.378+0000 7f291b7fe700 1 -- 172.21.15.50:0/2835688018 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f291c032020 con 0x7f292c109e50 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 msgr2=0x7f292c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 --2- 172.21.15.50:0/2835688018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f291c003f20 tx=0x7f291c011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 shutdown_connections 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 --2- 172.21.15.50:0/2835688018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f292c109e50 0x7f292c19d670 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 >> 172.21.15.50:0/2835688018 conn(0x7f292c103a40 msgr2=0x7f292c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 shutdown_connections 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:29.379+0000 7f2930b0c700 1 -- 172.21.15.50:0/2835688018 wait complete. 2024-05-17T04:23:29.928 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-17T04:23:29.951 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:23:29.951 INFO:teuthology.orchestra.run.smithi050.stdout:Creating mgr... 2024-05-17T04:23:29.951 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-17T04:23:29.952 INFO:teuthology.orchestra.run.smithi050.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-17T04:23:30.204 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T04:23:30.204 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-17T04:23:30.355 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968.target.wants/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@.service. 2024-05-17T04:23:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:30 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2835688018' entity='client.admin' 2024-05-17T04:23:30.847 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:30 smithi050 podman[31948]: 2024-05-17 04:23:30.775499982 +0000 UTC m=+0.295561575 container start 5b7105165780138bc00349df9629873575e353521f012af119cba9151fbc6070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T04:23:30.848 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:30 smithi050 bash[31948]: 5b7105165780138bc00349df9629873575e353521f012af119cba9151fbc6070 2024-05-17T04:23:30.862 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T04:23:30.863 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-17T04:23:30.870 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T04:23:30.870 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:23:30.871 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-17T04:23:30.871 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to enable service . firewalld.service is not available 2024-05-17T04:23:30.879 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T04:23:30.879 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr to start... 2024-05-17T04:23:30.887 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr... 2024-05-17T04:23:31.104 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:30 smithi050 systemd[1]: Started Ceph mgr.x for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:23:31.104 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:30.950+0000 7f34d6cd4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T04:23:31.382 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:31.147+0000 7f34d6cd4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:23:31.382 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:31.255+0000 7f34d6cd4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T04:23:31.681 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:31 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2058789908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T04:23:31.682 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:31.590+0000 7f34d6cd4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T04:23:31.682 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:31.681+0000 7f34d6cd4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T04:23:31.982 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T04:23:31.985 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T04:23:31.986 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:31.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T04:23:31.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T04:23:31.987 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T04:23:31.995 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T04:23:31.995 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T04:23:31.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T04:23:31.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T04:23:23.371760+0000", 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.299+0000 7f896a2f3700 1 Processor -- start 2024-05-17T04:23:31.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.299+0000 7f896a2f3700 1 -- start start 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.299+0000 7f896a2f3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.299+0000 7f896a2f3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f896407cde0 con 0x7f8964109e40 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.300+0000 7f8963fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.300+0000 7f8963fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44914/0 (socket says 172.21.15.50:44914) 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.300+0000 7f8963fff700 1 -- 172.21.15.50:0/3194493446 learned_addr learned my addr 172.21.15.50:0/3194493446 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.300+0000 7f8963fff700 1 -- 172.21.15.50:0/3194493446 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896410c7a0 con 0x7f8964109e40 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.300+0000 7f8963fff700 1 --2- 172.21.15.50:0/3194493446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89540098a0 tx=0x7f8954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4763da943bb83a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.301+0000 7f8962ffd700 1 -- 172.21.15.50:0/3194493446 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8954004160 con 0x7f8964109e40 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.301+0000 7f8962ffd700 1 -- 172.21.15.50:0/3194493446 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8954004300 con 0x7f8964109e40 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.301+0000 7f8962ffd700 1 -- 172.21.15.50:0/3194493446 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89540119e0 con 0x7f8964109e40 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.301+0000 7f896a2f3700 1 -- 172.21.15.50:0/3194493446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 msgr2=0x7f896410c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:31.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.301+0000 7f896a2f3700 1 --2- 172.21.15.50:0/3194493446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89540098a0 tx=0x7f8954009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 -- 172.21.15.50:0/3194493446 shutdown_connections 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 --2- 172.21.15.50:0/3194493446 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896410c260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 -- 172.21.15.50:0/3194493446 >> 172.21.15.50:0/3194493446 conn(0x7f8964103af0 msgr2=0x7f8964105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 -- 172.21.15.50:0/3194493446 shutdown_connections 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 -- 172.21.15.50:0/3194493446 wait complete. 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 Processor -- start 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.302+0000 7f896a2f3700 1 -- start start 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f896a2f3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f896a2f3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f896410d6a0 con 0x7f8964109e40 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f8963fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f8963fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44918/0 (socket says 172.21.15.50:44918) 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f8963fff700 1 -- 172.21.15.50:0/2058789908 learned_addr learned my addr 172.21.15.50:0/2058789908 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:32.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.303+0000 7f8963fff700 1 -- 172.21.15.50:0/2058789908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896419dba0 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f8963fff700 1 --2- 172.21.15.50:0/2058789908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8954003f20 tx=0x7f8954011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89540038b0 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8954003a50 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896419de30 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8954019550 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a0a40 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8954019b90 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.304+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8954024730 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.305+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896410a250 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.307+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8954043ec0 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.381+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f89641a1030 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.382+0000 7f89617fa700 1 -- 172.21.15.50:0/2058789908 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8954032030 con 0x7f8964109e40 2024-05-17T04:23:32.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 msgr2=0x7f896419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 --2- 172.21.15.50:0/2058789908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8954003f20 tx=0x7f8954011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 shutdown_connections 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 --2- 172.21.15.50:0/2058789908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8964109e40 0x7f896419d660 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 >> 172.21.15.50:0/2058789908 conn(0x7f8964103af0 msgr2=0x7f89641054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 shutdown_connections 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:31.383+0000 7f896a2f3700 1 -- 172.21.15.50:0/2058789908 wait complete. 2024-05-17T04:23:32.002 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (1/15)... 2024-05-17T04:23:33.310 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:32 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:32.955+0000 7f34d6cd4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T04:23:33.310 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:33.132+0000 7f34d6cd4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T04:23:33.310 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:33.309+0000 7f34d6cd4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T04:23:33.917 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:33.610+0000 7f34d6cd4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T04:23:33.918 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:33.706+0000 7f34d6cd4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T04:23:33.918 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:33.917+0000 7f34d6cd4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T04:23:34.251 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:34.120+0000 7f34d6cd4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:23:34.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:34 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1474058175' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T04:23:34.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:34.769+0000 7f34d6cd4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T04:23:35.086 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-17T04:23:35.087 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T04:23:35.088 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T04:23:35.089 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T04:23:35.090 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T04:23:23.371760+0000", 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T04:23:35.091 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.462+0000 7f9d3392f700 1 Processor -- start 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d3392f700 1 -- start start 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d3392f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d3392f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c07cde0 con 0x7f9d2c109e40 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d316cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d316cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44920/0 (socket says 172.21.15.50:44920) 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d316cb700 1 -- 172.21.15.50:0/1535790931 learned_addr learned my addr 172.21.15.50:0/1535790931 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d316cb700 1 -- 172.21.15.50:0/1535790931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c10c7a0 con 0x7f9d2c109e40 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.463+0000 7f9d316cb700 1 --2- 172.21.15.50:0/1535790931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d280098a0 tx=0x7f9d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d978972d95d031 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:35.092 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.464+0000 7f9d23fff700 1 -- 172.21.15.50:0/1535790931 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d28004160 con 0x7f9d2c109e40 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.464+0000 7f9d23fff700 1 -- 172.21.15.50:0/1535790931 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9d28004300 con 0x7f9d2c109e40 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.464+0000 7f9d23fff700 1 -- 172.21.15.50:0/1535790931 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d280119e0 con 0x7f9d2c109e40 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 -- 172.21.15.50:0/1535790931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 msgr2=0x7f9d2c10c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 --2- 172.21.15.50:0/1535790931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d280098a0 tx=0x7f9d28009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 -- 172.21.15.50:0/1535790931 shutdown_connections 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 --2- 172.21.15.50:0/1535790931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c10c260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 -- 172.21.15.50:0/1535790931 >> 172.21.15.50:0/1535790931 conn(0x7f9d2c103af0 msgr2=0x7f9d2c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 -- 172.21.15.50:0/1535790931 shutdown_connections 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.465+0000 7f9d3392f700 1 -- 172.21.15.50:0/1535790931 wait complete. 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d3392f700 1 Processor -- start 2024-05-17T04:23:35.093 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d3392f700 1 -- start start 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d3392f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d3392f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c10d6a0 con 0x7f9d2c109e40 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d316cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d316cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44932/0 (socket says 172.21.15.50:44932) 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d316cb700 1 -- 172.21.15.50:0/1474058175 learned_addr learned my addr 172.21.15.50:0/1474058175 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.466+0000 7f9d316cb700 1 -- 172.21.15.50:0/1474058175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c19dba0 con 0x7f9d2c109e40 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d316cb700 1 --2- 172.21.15.50:0/1474058175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d280116a0 tx=0x7f9d28011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d28003970 con 0x7f9d2c109e40 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9d28003b10 con 0x7f9d2c109e40 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c19de30 con 0x7f9d2c109e40 2024-05-17T04:23:35.094 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d28019550 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c1a0a40 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d28019bc0 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.467+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d2800b720 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.468+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2c10a250 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.470+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9d2801aa50 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.544+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9d2c1a1030 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.544+0000 7f9d227fc700 1 -- 172.21.15.50:0/1474058175 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9d2801a430 con 0x7f9d2c109e40 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 msgr2=0x7f9d2c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 --2- 172.21.15.50:0/1474058175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d280116a0 tx=0x7f9d28011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 shutdown_connections 2024-05-17T04:23:35.095 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 --2- 172.21.15.50:0/1474058175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9d2c109e40 0x7f9d2c19d660 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:35.096 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 >> 172.21.15.50:0/1474058175 conn(0x7f9d2c103af0 msgr2=0x7f9d2c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:35.096 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 shutdown_connections 2024-05-17T04:23:35.096 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:34.545+0000 7f9d3392f700 1 -- 172.21.15.50:0/1474058175 wait complete. 2024-05-17T04:23:35.096 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (2/15)... 2024-05-17T04:23:35.326 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:35.004+0000 7f34d6cd4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T04:23:35.826 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:35.449+0000 7f34d6cd4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T04:23:35.826 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:35.549+0000 7f34d6cd4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T04:23:36.307 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:35.933+0000 7f34d6cd4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T04:23:36.307 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:36.307+0000 7f34d6cd4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T04:23:36.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:36.403+0000 7f34d6cd4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T04:23:37.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:36.656+0000 7f34d6cd4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T04:23:37.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:36.795+0000 7f34d6cd4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T04:23:37.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:36.895+0000 7f34d6cd4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T04:23:37.456 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:37.237+0000 7f34d6cd4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T04:23:37.456 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:37.334+0000 7f34d6cd4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T04:23:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:37 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/388780458' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T04:23:38.277 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:37.983+0000 7f34d6cd4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T04:23:38.277 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:38.210+0000 7f34d6cd4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T04:23:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:38.291 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T04:23:38.292 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T04:23:38.293 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T04:23:23.371760+0000", 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcaec2f700 1 Processor -- start 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcaec2f700 1 -- start start 2024-05-17T04:23:38.294 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcaec2f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcaec2f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdca807cde0 con 0x7fdca8109e40 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcac9cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcac9cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44936/0 (socket says 172.21.15.50:44936) 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.568+0000 7fdcac9cb700 1 -- 172.21.15.50:0/1535691920 learned_addr learned my addr 172.21.15.50:0/1535691920 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.569+0000 7fdcac9cb700 1 -- 172.21.15.50:0/1535691920 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca810c7a0 con 0x7fdca8109e40 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.569+0000 7fdcac9cb700 1 --2- 172.21.15.50:0/1535691920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098a0 tx=0x7fdc9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec37f386e9f1e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.569+0000 7fdca77fe700 1 -- 172.21.15.50:0/1535691920 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c004160 con 0x7fdca8109e40 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.569+0000 7fdca77fe700 1 -- 172.21.15.50:0/1535691920 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc9c004300 con 0x7fdca8109e40 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.569+0000 7fdca77fe700 1 -- 172.21.15.50:0/1535691920 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c0119e0 con 0x7fdca8109e40 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 -- 172.21.15.50:0/1535691920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 msgr2=0x7fdca810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 --2- 172.21.15.50:0/1535691920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098a0 tx=0x7fdc9c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 -- 172.21.15.50:0/1535691920 shutdown_connections 2024-05-17T04:23:38.295 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 --2- 172.21.15.50:0/1535691920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca810c260 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 -- 172.21.15.50:0/1535691920 >> 172.21.15.50:0/1535691920 conn(0x7fdca8103af0 msgr2=0x7fdca8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 -- 172.21.15.50:0/1535691920 shutdown_connections 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.570+0000 7fdcaec2f700 1 -- 172.21.15.50:0/1535691920 wait complete. 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcaec2f700 1 Processor -- start 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcaec2f700 1 -- start start 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcaec2f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcaec2f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdca810d6a0 con 0x7fdca8109e40 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcac9cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcac9cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44948/0 (socket says 172.21.15.50:44948) 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.571+0000 7fdcac9cb700 1 -- 172.21.15.50:0/388780458 learned_addr learned my addr 172.21.15.50:0/388780458 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdcac9cb700 1 -- 172.21.15.50:0/388780458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca8109b00 con 0x7fdca8109e40 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdcac9cb700 1 --2- 172.21.15.50:0/388780458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003f20 tx=0x7fdc9c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:38.296 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c0038b0 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca8107d50 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdc9c003a50 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c019550 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca8108230 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.572+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc9c019b90 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.573+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc9c024a30 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.573+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca810a250 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.576+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdc9c043ec0 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.650+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdca81a3710 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.650+0000 7fdca5ffb700 1 -- 172.21.15.50:0/388780458 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdc9c032030 con 0x7fdca8109e40 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 msgr2=0x7fdca81095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 --2- 172.21.15.50:0/388780458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003f20 tx=0x7fdc9c011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 shutdown_connections 2024-05-17T04:23:38.297 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 --2- 172.21.15.50:0/388780458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdca8109e40 0x7fdca81095c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:38.298 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 >> 172.21.15.50:0/388780458 conn(0x7fdca8103af0 msgr2=0x7fdca81054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:38.298 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 shutdown_connections 2024-05-17T04:23:38.298 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:37.651+0000 7fdcaec2f700 1 -- 172.21.15.50:0/388780458 wait complete. 2024-05-17T04:23:38.298 INFO:teuthology.orchestra.run.smithi050.stdout:mgr not available, waiting (3/15)... 2024-05-17T04:23:39.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: Activating manager daemon x 2024-05-17T04:23:39.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: mgrmap e2: x(active, starting, since 0.00381146s) 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: Manager daemon x is now available 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' 2024-05-17T04:23:39.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:38 smithi050 ceph-mon[31739]: from='mgr.14100 172.21.15.50:0/637815868' entity='mgr.x' 2024-05-17T04:23:40.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:40 smithi050 ceph-mon[31739]: mgrmap e3: x(active, since 1.00808s) 2024-05-17T04:23:41.480 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:41 smithi050 ceph-mon[31739]: mgrmap e4: x(active, since 2s) 2024-05-17T04:23:41.481 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:41 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3189007511' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-17T04:23:41.621 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "health": { 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 0 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-17T04:23:41.622 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "a" 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-17T04:23:41.623 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-17T04:23:41.632 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-17T04:23:41.633 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "restful" 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ], 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "modified": "2024-05-17T04:23:23.371760+0000", 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout }, 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:41.634 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe047912700 1 Processor -- start 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe047912700 1 -- start start 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe047912700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe047912700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe04007d650 con 0x7fe040109e60 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe0456ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe0456ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40316/0 (socket says 172.21.15.50:40316) 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.779+0000 7fe0456ae700 1 -- 172.21.15.50:0/3096360981 learned_addr learned my addr 172.21.15.50:0/3096360981 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.780+0000 7fe0456ae700 1 -- 172.21.15.50:0/3096360981 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04010c7c0 con 0x7fe040109e60 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.780+0000 7fe0456ae700 1 --2- 172.21.15.50:0/3096360981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0098a0 tx=0x7fe03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd31adf71c64bf4d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.780+0000 7fe037fff700 1 -- 172.21.15.50:0/3096360981 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe03c004160 con 0x7fe040109e60 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.780+0000 7fe037fff700 1 -- 172.21.15.50:0/3096360981 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe03c004300 con 0x7fe040109e60 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.780+0000 7fe037fff700 1 -- 172.21.15.50:0/3096360981 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe03c0119e0 con 0x7fe040109e60 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 -- 172.21.15.50:0/3096360981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 msgr2=0x7fe04010c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 --2- 172.21.15.50:0/3096360981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0098a0 tx=0x7fe03c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:41.635 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 -- 172.21.15.50:0/3096360981 shutdown_connections 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 --2- 172.21.15.50:0/3096360981 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04010c280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 -- 172.21.15.50:0/3096360981 >> 172.21.15.50:0/3096360981 conn(0x7fe040103af0 msgr2=0x7fe040105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 -- 172.21.15.50:0/3096360981 shutdown_connections 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.781+0000 7fe047912700 1 -- 172.21.15.50:0/3096360981 wait complete. 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe047912700 1 Processor -- start 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe047912700 1 -- start start 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe047912700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe047912700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe04010d6c0 con 0x7fe040109e60 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe0456ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe0456ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40322/0 (socket says 172.21.15.50:40322) 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.782+0000 7fe0456ae700 1 -- 172.21.15.50:0/3189007511 learned_addr learned my addr 172.21.15.50:0/3189007511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe0456ae700 1 -- 172.21.15.50:0/3189007511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019db90 con 0x7fe040109e60 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe0456ae700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe03c003f20 tx=0x7fe03c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:41.636 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe03c0038b0 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe03c003a50 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019de20 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe03c019550 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.783+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe040107c50 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.784+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fe03c019800 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.784+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04010a270 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.784+0000 7fe0367fc700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 0x7fe02c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.784+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe03c053b10 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.785+0000 7fe044ead700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 0x7fe02c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.785+0000 7fe044ead700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 0x7fe02c040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe030006fd0 tx=0x7fe030006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:40.790+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe03c017e90 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.090+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe040108700 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.091+0000 7fe0367fc700 1 -- 172.21.15.50:0/3189007511 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe03c023390 con 0x7fe040109e60 2024-05-17T04:23:41.637 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 msgr2=0x7fe02c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 0x7fe02c040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe030006fd0 tx=0x7fe030006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 msgr2=0x7fe04019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe03c003f20 tx=0x7fe03c011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 shutdown_connections 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7fe02c03e420 0x7fe02c040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 --2- 172.21.15.50:0/3189007511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe040109e60 0x7fe04019d650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 >> 172.21.15.50:0/3189007511 conn(0x7fe040103af0 msgr2=0x7fe0401054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 shutdown_connections 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:41.094+0000 7fe047912700 1 -- 172.21.15.50:0/3189007511 wait complete. 2024-05-17T04:23:41.638 INFO:teuthology.orchestra.run.smithi050.stdout:mgr is available 2024-05-17T04:23:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:42 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2786566452' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-17T04:23:42.843 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:42.843 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [global] 2024-05-17T04:23:42.843 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout fsid = 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:42.843 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-17T04:23:42.843 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 2024-05-17T04:23:42.844 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout [osd] 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.074+0000 7f483b234700 1 Processor -- start 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.074+0000 7f483b234700 1 -- start start 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.074+0000 7f483b234700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:42.845 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.074+0000 7f483b234700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f483410a1e0 con 0x7f4834109800 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f4838fd0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f4838fd0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40332/0 (socket says 172.21.15.50:40332) 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f4838fd0700 1 -- 172.21.15.50:0/988185141 learned_addr learned my addr 172.21.15.50:0/988185141 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f4838fd0700 1 -- 172.21.15.50:0/988185141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4834108480 con 0x7f4834109800 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f4838fd0700 1 --2- 172.21.15.50:0/988185141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f483000bb40 tx=0x7f483000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276d9c2d7c9b1965 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f482b7fe700 1 -- 172.21.15.50:0/988185141 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f483000dbe0 con 0x7f4834109800 2024-05-17T04:23:42.846 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f482b7fe700 1 -- 172.21.15.50:0/988185141 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f48300044a0 con 0x7f4834109800 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.075+0000 7f482b7fe700 1 -- 172.21.15.50:0/988185141 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4830003e30 con 0x7f4834109800 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 -- 172.21.15.50:0/988185141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 msgr2=0x7f4834109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 --2- 172.21.15.50:0/988185141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f483000bb40 tx=0x7f483000be50 comp rx=0 tx=0).stop 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 -- 172.21.15.50:0/988185141 shutdown_connections 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 --2- 172.21.15.50:0/988185141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f4834109c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 -- 172.21.15.50:0/988185141 >> 172.21.15.50:0/988185141 conn(0x7f4834103270 msgr2=0x7f48341056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 -- 172.21.15.50:0/988185141 shutdown_connections 2024-05-17T04:23:42.847 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.076+0000 7f483b234700 1 -- 172.21.15.50:0/988185141 wait complete. 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f483b234700 1 Processor -- start 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f483b234700 1 -- start start 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f483b234700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f483b234700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4834107370 con 0x7f4834109800 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f4838fd0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f4838fd0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40336/0 (socket says 172.21.15.50:40336) 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.077+0000 7f4838fd0700 1 -- 172.21.15.50:0/2786566452 learned_addr learned my addr 172.21.15.50:0/2786566452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:42.848 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f4838fd0700 1 -- 172.21.15.50:0/2786566452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483419fda0 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f4838fd0700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4830003bc0 tx=0x7f4830003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4830010580 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f483001d070 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48341a0030 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48300219e0 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.078+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a2c40 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.079+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f483001f030 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.079+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4834109c10 con 0x7f4834109800 2024-05-17T04:23:42.849 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.079+0000 7f4829ffb700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 0x7f481c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.079+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f48300535a0 con 0x7f4834109800 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.080+0000 7f482bfff700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 0x7f481c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.080+0000 7f482bfff700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 0x7f481c040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.084+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4830015070 con 0x7f4834109800 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.294+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f48341a4890 con 0x7f4834109800 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.296+0000 7f4829ffb700 1 -- 172.21.15.50:0/2786566452 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f4830017300 con 0x7f4834109800 2024-05-17T04:23:42.850 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 msgr2=0x7f481c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 0x7f481c040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4824006fd0 tx=0x7f4824006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 msgr2=0x7f483419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4830003bc0 tx=0x7f4830003f80 comp rx=0 tx=0).stop 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 shutdown_connections 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f481c03e470 0x7f481c040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 --2- 172.21.15.50:0/2786566452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4834109800 0x7f483419f860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 >> 172.21.15.50:0/2786566452 conn(0x7f4834103270 msgr2=0x7f48341916b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:42.851 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 shutdown_connections 2024-05-17T04:23:42.852 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:42.298+0000 7f483b234700 1 -- 172.21.15.50:0/2786566452 wait complete. 2024-05-17T04:23:42.852 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling cephadm module... 2024-05-17T04:23:43.802 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:43 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/209866505' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-17T04:23:43.802 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: ignoring --setuser ceph since I am not root 2024-05-17T04:23:43.802 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: ignoring --setgroup ceph since I am not root 2024-05-17T04:23:44.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:43.801+0000 7fb99e379200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T04:23:44.077 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:44.002+0000 7fb99e379200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7daae69700 1 Processor -- start 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7daae69700 1 -- start start 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7daae69700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7daae69700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7da407d720 con 0x7f7da4109ed0 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7da8c05700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7da8c05700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40340/0 (socket says 172.21.15.50:40340) 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.255+0000 7f7da8c05700 1 -- 172.21.15.50:0/561591023 learned_addr learned my addr 172.21.15.50:0/561591023 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.256+0000 7f7da8c05700 1 -- 172.21.15.50:0/561591023 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da410c830 con 0x7f7da4109ed0 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.256+0000 7f7da8c05700 1 --2- 172.21.15.50:0/561591023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d980098a0 tx=0x7f7d98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec74d4ed22be9830 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.256+0000 7f7da37fe700 1 -- 172.21.15.50:0/561591023 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d98004160 con 0x7f7da4109ed0 2024-05-17T04:23:44.172 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.256+0000 7f7da37fe700 1 -- 172.21.15.50:0/561591023 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7d98004300 con 0x7f7da4109ed0 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.256+0000 7f7da37fe700 1 -- 172.21.15.50:0/561591023 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d980119e0 con 0x7f7da4109ed0 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 -- 172.21.15.50:0/561591023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 msgr2=0x7f7da410c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 --2- 172.21.15.50:0/561591023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d980098a0 tx=0x7f7d98009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 -- 172.21.15.50:0/561591023 shutdown_connections 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 --2- 172.21.15.50:0/561591023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da410c2f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 -- 172.21.15.50:0/561591023 >> 172.21.15.50:0/561591023 conn(0x7f7da4103a40 msgr2=0x7f7da4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 -- 172.21.15.50:0/561591023 shutdown_connections 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 -- 172.21.15.50:0/561591023 wait complete. 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.257+0000 7f7daae69700 1 Processor -- start 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7daae69700 1 -- start start 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7daae69700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7daae69700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410d6d0 con 0x7f7da4109ed0 2024-05-17T04:23:44.173 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da8c05700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da8c05700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40342/0 (socket says 172.21.15.50:40342) 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da8c05700 1 -- 172.21.15.50:0/209866505 learned_addr learned my addr 172.21.15.50:0/209866505 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da8c05700 1 -- 172.21.15.50:0/209866505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419dbd0 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da8c05700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d98003f20 tx=0x7f7d98011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.258+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d980038b0 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.259+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7d98003a50 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.259+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419de60 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.259+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d98019550 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.259+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a0a70 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.260+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f7d98019850 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.260+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da410a2e0 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.260+0000 7f7da1ffb700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 0x7f7d940408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.260+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d98053c90 con 0x7f7da4109ed0 2024-05-17T04:23:44.174 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.260+0000 7f7da3fff700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 0x7f7d940408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.261+0000 7f7da3fff700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 0x7f7d940408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.265+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d9802ce20 con 0x7f7da4109ed0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.544+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f7da41a1510 con 0x7f7da4109ed0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.596+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7d98023390 con 0x7f7da4109ed0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.598+0000 7f7da1ffb700 1 -- 172.21.15.50:0/209866505 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7d9802d440 con 0x7f7da4109ed0 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 msgr2=0x7f7d940408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 0x7f7d940408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 msgr2=0x7f7da419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d98003f20 tx=0x7f7d98011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 shutdown_connections 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f7d9403e3d0 0x7f7d940408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 --2- 172.21.15.50:0/209866505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7da4109ed0 0x7f7da419d690 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:44.175 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.599+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 >> 172.21.15.50:0/209866505 conn(0x7f7da4103a40 msgr2=0x7f7da41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:44.176 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.600+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 shutdown_connections 2024-05-17T04:23:44.176 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:43.600+0000 7f7daae69700 1 -- 172.21.15.50:0/209866505 wait complete. 2024-05-17T04:23:44.390 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:44.109+0000 7fb99e379200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T04:23:44.680 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:44 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/209866505' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-17T04:23:44.680 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:44 smithi050 ceph-mon[31739]: mgrmap e5: x(active, since 5s) 2024-05-17T04:23:44.680 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:44.446+0000 7fb99e379200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T04:23:44.680 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:44.540+0000 7fb99e379200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T04:23:45.458 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.615+0000 7f37380db700 1 Processor -- start 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f37380db700 1 -- start start 2024-05-17T04:23:45.464 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f37380db700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f37380db700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f373010a1d0 con 0x7f37301097f0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f3735e77700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f3735e77700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40362/0 (socket says 172.21.15.50:40362) 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.616+0000 7f3735e77700 1 -- 172.21.15.50:0/1787679220 learned_addr learned my addr 172.21.15.50:0/1787679220 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.617+0000 7f3735e77700 1 -- 172.21.15.50:0/1787679220 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373010a350 con 0x7f37301097f0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.617+0000 7f3735e77700 1 --2- 172.21.15.50:0/1787679220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f372c0098a0 tx=0x7f372c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fd244b72c079ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.617+0000 7f3734e75700 1 -- 172.21.15.50:0/1787679220 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c004160 con 0x7f37301097f0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.617+0000 7f3734e75700 1 -- 172.21.15.50:0/1787679220 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f372c004300 con 0x7f37301097f0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.617+0000 7f3734e75700 1 -- 172.21.15.50:0/1787679220 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c0119e0 con 0x7f37301097f0 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 -- 172.21.15.50:0/1787679220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 msgr2=0x7f3730109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 --2- 172.21.15.50:0/1787679220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f372c0098a0 tx=0x7f372c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 -- 172.21.15.50:0/1787679220 shutdown_connections 2024-05-17T04:23:45.465 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 --2- 172.21.15.50:0/1787679220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f3730109c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 -- 172.21.15.50:0/1787679220 >> 172.21.15.50:0/1787679220 conn(0x7f3730103270 msgr2=0x7f37301056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 -- 172.21.15.50:0/1787679220 shutdown_connections 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.618+0000 7f37380db700 1 -- 172.21.15.50:0/1787679220 wait complete. 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f37380db700 1 Processor -- start 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f37380db700 1 -- start start 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f37380db700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f37380db700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3730108150 con 0x7f37301097f0 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f3735e77700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f3735e77700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40376/0 (socket says 172.21.15.50:40376) 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f3735e77700 1 -- 172.21.15.50:0/1620717304 learned_addr learned my addr 172.21.15.50:0/1620717304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.619+0000 7f3735e77700 1 -- 172.21.15.50:0/1620717304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373019fd70 con 0x7f37301097f0 2024-05-17T04:23:45.466 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f3735e77700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f372c003f20 tx=0x7f372c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c0038b0 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f372c003a50 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c0195a0 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301a0000 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.620+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301a2c10 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.621+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f372c019860 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.621+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3730109c00 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.621+0000 7f3726ffd700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 0x7f371c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.622+0000 7f3735676700 1 -- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 msgr2=0x7f371c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.622+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f372c053bd0 con 0x7f37301097f0 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.622+0000 7f3735676700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 0x7f371c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:23:45.467 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.631+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f372c017e90 con 0x7f37301097f0 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.823+0000 7f3735676700 1 -- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 msgr2=0x7f371c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.823+0000 7f3735676700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 0x7f371c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.908+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3730198a20 con 0x7f37301097f0 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.908+0000 7f3726ffd700 1 -- 172.21.15.50:0/1620717304 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3730198a20 con 0x7f37301097f0 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 msgr2=0x7f371c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 0x7f371c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 msgr2=0x7f373019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f372c003f20 tx=0x7f372c011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 shutdown_connections 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f371c03e420 0x7f371c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 --2- 172.21.15.50:0/1620717304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37301097f0 0x7f373019f830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:45.468 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 >> 172.21.15.50:0/1620717304 conn(0x7f3730103270 msgr2=0x7f3730195ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:45.469 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 shutdown_connections 2024-05-17T04:23:45.469 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:44.911+0000 7f37380db700 1 -- 172.21.15.50:0/1620717304 wait complete. 2024-05-17T04:23:45.469 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-05-17T04:23:45.469 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 5... 2024-05-17T04:23:45.878 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:45 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1620717304' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T04:23:46.204 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:45.877+0000 7fb99e379200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T04:23:46.204 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:46.066+0000 7fb99e379200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T04:23:46.552 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:46.245+0000 7fb99e379200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T04:23:46.552 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:46.551+0000 7fb99e379200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T04:23:46.826 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:46.650+0000 7fb99e379200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T04:23:47.326 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:46.863+0000 7fb99e379200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T04:23:47.326 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:47.067+0000 7fb99e379200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:23:48.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:47.701+0000 7fb99e379200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T04:23:48.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:47.935+0000 7fb99e379200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T04:23:48.826 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:48.383+0000 7fb99e379200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T04:23:48.826 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:48.486+0000 7fb99e379200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T04:23:49.255 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:48.876+0000 7fb99e379200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T04:23:49.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:49.255+0000 7fb99e379200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T04:23:49.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:49.353+0000 7fb99e379200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T04:23:50.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:49.612+0000 7fb99e379200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T04:23:50.077 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:49.752+0000 7fb99e379200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T04:23:50.077 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:49.855+0000 7fb99e379200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T04:23:50.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:50.196+0000 7fb99e379200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T04:23:50.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:50.295+0000 7fb99e379200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T04:23:51.215 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:50.933+0000 7fb99e379200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T04:23:51.216 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:23:51.162+0000 7fb99e379200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T04:23:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: Active manager daemon x restarted 2024-05-17T04:23:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: Activating manager daemon x 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: osdmap e2: 0 total, 0 up, 0 in 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: mgrmap e6: x(active, starting, since 0.0040529s) 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: Manager daemon x is now available 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:23:51.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:51 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:23:52.494 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:52 smithi050 ceph-mon[31739]: Found migration_current of "None". Setting to last migration. 2024-05-17T04:23:52.494 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:52 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T04:23:52.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:52 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T04:23:52.495 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:52 smithi050 ceph-mon[31739]: mgrmap e7: x(active, since 1.00695s) 2024-05-17T04:23:52.796 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.996+0000 7f07d3955700 1 Processor -- start 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d3955700 1 -- start start 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d3955700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:52.797 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d3955700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc07cde0 con 0x7f07cc109e40 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d16f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d16f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40384/0 (socket says 172.21.15.50:40384) 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d16f1700 1 -- 172.21.15.50:0/1686915902 learned_addr learned my addr 172.21.15.50:0/1686915902 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.997+0000 7f07d16f1700 1 -- 172.21.15.50:0/1686915902 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc10c7a0 con 0x7f07cc109e40 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.998+0000 7f07d16f1700 1 --2- 172.21.15.50:0/1686915902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07c00098a0 tx=0x7f07c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c2075451d76b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.998+0000 7f07bffff700 1 -- 172.21.15.50:0/1686915902 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07c0004160 con 0x7f07cc109e40 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.998+0000 7f07bffff700 1 -- 172.21.15.50:0/1686915902 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f07c0004300 con 0x7f07cc109e40 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.998+0000 7f07bffff700 1 -- 172.21.15.50:0/1686915902 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07c00119e0 con 0x7f07cc109e40 2024-05-17T04:23:52.798 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 -- 172.21.15.50:0/1686915902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 msgr2=0x7f07cc10c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 --2- 172.21.15.50:0/1686915902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07c00098a0 tx=0x7f07c0009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 -- 172.21.15.50:0/1686915902 shutdown_connections 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 --2- 172.21.15.50:0/1686915902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc10c260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 -- 172.21.15.50:0/1686915902 >> 172.21.15.50:0/1686915902 conn(0x7f07cc103af0 msgr2=0x7f07cc105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 -- 172.21.15.50:0/1686915902 shutdown_connections 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 -- 172.21.15.50:0/1686915902 wait complete. 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:45.999+0000 7f07d3955700 1 Processor -- start 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d3955700 1 -- start start 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d3955700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d3955700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10d6a0 con 0x7f07cc109e40 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d16f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d16f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40396/0 (socket says 172.21.15.50:40396) 2024-05-17T04:23:52.799 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d16f1700 1 -- 172.21.15.50:0/2293857404 learned_addr learned my addr 172.21.15.50:0/2293857404 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d16f1700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc19fdb0 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.000+0000 7f07d16f1700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07c0003f20 tx=0x7f07c0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.001+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07c00038b0 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.001+0000 7f07d3955700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1a0040 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.001+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f07c0003a50 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.001+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07c0019550 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.001+0000 7f07d3955700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a2c50 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f07c001aa30 con 0x7f07cc109e40 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07be7fc700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:52.800 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07b8041040 con 0x7f07b803e420 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f07c0053b30 con 0x7f07cc109e40 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07d0ef0700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.003+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.204+0000 7f07d0ef0700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.204+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.605+0000 7f07d0ef0700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:46.605+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T04:23:52.801 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:47.406+0000 7f07d0ef0700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:47.406+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:49.007+0000 7f07d0ef0700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/193225580 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:49.007+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:51.166+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f07c001a430 con 0x7f07cc109e40 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:51.166+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 msgr2=0x7f07b8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:51.166+0000 7f07be7fc700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.167+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f07c0055af0 con 0x7f07cc109e40 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.168+0000 7f07be7fc700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 0x7f07b80441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.168+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07c002cde0 con 0x7f07b8041db0 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.168+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 0x7f07b80441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.169+0000 7f07d0ef0700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 0x7f07b80441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07c8003a10 tx=0x7f07c8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:52.802 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.170+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f07c002cde0 con 0x7f07b8041db0 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.178+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f07cc10a250 con 0x7f07b8041db0 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.178+0000 7f07be7fc700 1 -- 172.21.15.50:0/2293857404 <== mgr.14120 v2:172.21.15.50:6800/2489127888 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f07cc10a250 con 0x7f07b8041db0 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 msgr2=0x7f07b80441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 0x7f07b80441d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07c8003a10 tx=0x7f07c8009290 comp rx=0 tx=0).stop 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 msgr2=0x7f07cc19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07c0003f20 tx=0x7f07c0011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 shutdown_connections 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f07b8041db0 0x7f07b80441d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07cc109e40 0x7f07cc19f870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 --2- 172.21.15.50:0/2293857404 >> [v2:172.21.15.50:6800/193225580,v1:172.21.15.50:6801/193225580] conn(0x7f07b803e420 0x7f07b8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 >> 172.21.15.50:0/2293857404 conn(0x7f07cc103af0 msgr2=0x7f07cc107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:52.803 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 shutdown_connections 2024-05-17T04:23:52.804 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:52.179+0000 7f07b7fff700 1 -- 172.21.15.50:0/2293857404 wait complete. 2024-05-17T04:23:52.804 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 5 is available 2024-05-17T04:23:52.804 INFO:teuthology.orchestra.run.smithi050.stdout:Setting orchestrator backend to cephadm... 2024-05-17T04:23:53.459 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:53 smithi050 ceph-mon[31739]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T04:23:53.459 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:53 smithi050 ceph-mon[31739]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T04:23:53.459 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:53 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:53.459 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:53 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.212+0000 7f84ba2a5700 1 Processor -- start 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.212+0000 7f84ba2a5700 1 -- start start 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84ba2a5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84ba2a5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84b407e120 con 0x7f84b407f7b0 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84b3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84b3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55864/0 (socket says 172.21.15.50:55864) 2024-05-17T04:23:54.059 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84b3fff700 1 -- 172.21.15.50:0/4051118593 learned_addr learned my addr 172.21.15.50:0/4051118593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84b3fff700 1 -- 172.21.15.50:0/4051118593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b407e2a0 con 0x7f84b407f7b0 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.213+0000 7f84b3fff700 1 --2- 172.21.15.50:0/4051118593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84a40098a0 tx=0x7f84a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f717b96290d2581 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84b2ffd700 1 -- 172.21.15.50:0/4051118593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84a4004160 con 0x7f84b407f7b0 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84b2ffd700 1 -- 172.21.15.50:0/4051118593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f84a4004300 con 0x7f84b407f7b0 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84b2ffd700 1 -- 172.21.15.50:0/4051118593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84a40119e0 con 0x7f84b407f7b0 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 -- 172.21.15.50:0/4051118593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 msgr2=0x7f84b407dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/4051118593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84a40098a0 tx=0x7f84a4009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 -- 172.21.15.50:0/4051118593 shutdown_connections 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/4051118593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b407dbe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 -- 172.21.15.50:0/4051118593 >> 172.21.15.50:0/4051118593 conn(0x7f84b4104b40 msgr2=0x7f84b4107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 -- 172.21.15.50:0/4051118593 shutdown_connections 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.214+0000 7f84ba2a5700 1 -- 172.21.15.50:0/4051118593 wait complete. 2024-05-17T04:23:54.060 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.215+0000 7f84ba2a5700 1 Processor -- start 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.215+0000 7f84ba2a5700 1 -- start start 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84ba2a5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84ba2a5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f84b410f890 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84b3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84b3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55872/0 (socket says 172.21.15.50:55872) 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84b3fff700 1 -- 172.21.15.50:0/3488307004 learned_addr learned my addr 172.21.15.50:0/3488307004 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84b3fff700 1 -- 172.21.15.50:0/3488307004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b419fdb0 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.216+0000 7f84b3fff700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84a40116a0 tx=0x7f84a4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.219+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84a4003970 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.219+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f84a4003b10 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.219+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84a4019550 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.219+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b41a0040 con 0x7f84b407f7b0 2024-05-17T04:23:54.061 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.219+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a2c50 con 0x7f84b407f7b0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.220+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f84a40197c0 con 0x7f84b407f7b0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.220+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b407dbe0 con 0x7f84b407f7b0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.221+0000 7f84b17fa700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 0x7f84a00408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.221+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f84a40536c0 con 0x7f84b407f7b0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.221+0000 7f84b37fe700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 0x7f84a00408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.221+0000 7f84b37fe700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 0x7f84a00408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f849c006fd0 tx=0x7f849c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.226+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84a402adc0 con 0x7f84b407f7b0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.458+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f84b4105c40 con 0x7f84a003e3a0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.463+0000 7f84b17fa700 1 -- 172.21.15.50:0/3488307004 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f84b4105c40 con 0x7f84a003e3a0 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 msgr2=0x7f84a00408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 0x7f84a00408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f849c006fd0 tx=0x7f849c006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 msgr2=0x7f84b419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:54.062 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84a40116a0 tx=0x7f84a4011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 shutdown_connections 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f84a003e3a0 0x7f84a00408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 --2- 172.21.15.50:0/3488307004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f84b407f7b0 0x7f84b419f870 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 >> 172.21.15.50:0/3488307004 conn(0x7f84b4104b40 msgr2=0x7f84b4105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 shutdown_connections 2024-05-17T04:23:54.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:53.466+0000 7f84ba2a5700 1 -- 172.21.15.50:0/3488307004 wait complete. 2024-05-17T04:23:54.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:54 smithi050 ceph-mon[31739]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:23:54.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:54 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:54.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:54 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:23:54.726 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:54 smithi050 ceph-mon[31739]: mgrmap e8: x(active, since 2s) 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa05af12700 1 Processor -- start 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa05af12700 1 -- start start 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa05af12700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa05af12700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa054108380 con 0x7fa0541079a0 2024-05-17T04:23:55.225 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa058cae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa058cae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55878/0 (socket says 172.21.15.50:55878) 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.501+0000 7fa058cae700 1 -- 172.21.15.50:0/1720906792 learned_addr learned my addr 172.21.15.50:0/1720906792 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.502+0000 7fa058cae700 1 -- 172.21.15.50:0/1720906792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa054108b70 con 0x7fa0541079a0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.502+0000 7fa058cae700 1 --2- 172.21.15.50:0/1720906792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77146709dce936 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.502+0000 7fa04b7fe700 1 -- 172.21.15.50:0/1720906792 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050004160 con 0x7fa0541079a0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.502+0000 7fa04b7fe700 1 -- 172.21.15.50:0/1720906792 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa050004300 con 0x7fa0541079a0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.502+0000 7fa04b7fe700 1 -- 172.21.15.50:0/1720906792 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500119e0 con 0x7fa0541079a0 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- 172.21.15.50:0/1720906792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 msgr2=0x7fa054107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 --2- 172.21.15.50:0/1720906792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098a0 tx=0x7fa050009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- 172.21.15.50:0/1720906792 shutdown_connections 2024-05-17T04:23:55.226 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 --2- 172.21.15.50:0/1720906792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa054107db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- 172.21.15.50:0/1720906792 >> 172.21.15.50:0/1720906792 conn(0x7fa054103270 msgr2=0x7fa0541056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- 172.21.15.50:0/1720906792 shutdown_connections 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- 172.21.15.50:0/1720906792 wait complete. 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 Processor -- start 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.503+0000 7fa05af12700 1 -- start start 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa05af12700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa05af12700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa054109290 con 0x7fa0541079a0 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa058cae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa058cae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55886/0 (socket says 172.21.15.50:55886) 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa058cae700 1 -- 172.21.15.50:0/603820239 learned_addr learned my addr 172.21.15.50:0/603820239 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa058cae700 1 -- 172.21.15.50:0/603820239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05419dbf0 con 0x7fa0541079a0 2024-05-17T04:23:55.227 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa058cae700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa050003f20 tx=0x7fa050011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500038b0 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa050003a50 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.504+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05419de80 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.505+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050019550 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.505+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a0a90 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa050019800 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0541098b0 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa049ffb700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 0x7fa03c040970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa050053c20 con 0x7fa0541079a0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa04bfff700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 0x7fa03c040970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:55.228 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.506+0000 7fa04bfff700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 0x7fa03c040970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.511+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa05002ce20 con 0x7fa0541079a0 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.725+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa054063c70 con 0x7fa03c03e460 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.726+0000 7fa049ffb700 1 -- 172.21.15.50:0/603820239 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa054063c70 con 0x7fa03c03e460 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 msgr2=0x7fa03c040970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 0x7fa03c040970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa044006fd0 tx=0x7fa044006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 msgr2=0x7fa05419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa050003f20 tx=0x7fa050011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 shutdown_connections 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa03c03e460 0x7fa03c040970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 --2- 172.21.15.50:0/603820239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0541079a0 0x7fa05419d6b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.728+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 >> 172.21.15.50:0/603820239 conn(0x7fa054103270 msgr2=0x7fa054191690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.729+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 shutdown_connections 2024-05-17T04:23:55.229 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:54.729+0000 7fa05af12700 1 -- 172.21.15.50:0/603820239 wait complete. 2024-05-17T04:23:55.230 INFO:teuthology.orchestra.run.smithi050.stdout:Generating ssh key... 2024-05-17T04:23:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:55 smithi050 ceph-mon[31739]: [17/May/2024:04:23:54] ENGINE Bus STARTING 2024-05-17T04:23:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:55 smithi050 ceph-mon[31739]: [17/May/2024:04:23:54] ENGINE Serving on https://172.21.15.50:7150 2024-05-17T04:23:55.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:55 smithi050 ceph-mon[31739]: [17/May/2024:04:23:54] ENGINE Serving on http://172.21.15.50:8765 2024-05-17T04:23:55.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:55 smithi050 ceph-mon[31739]: [17/May/2024:04:23:54] ENGINE Bus STARTED 2024-05-17T04:23:55.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:55 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:23:56.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:56 smithi050 ceph-mon[31739]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: Generating public/private rsa key pair. 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: Your identification has been saved in /tmp/tmpv0giemhb/key. 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: Your public key has been saved in /tmp/tmpv0giemhb/key.pub. 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: The key fingerprint is: 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: SHA256:nbVd2XqIcl/0gdNA5jNHqVwSV7RQoK0HxHdXbgLx3/M ceph-0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: The key's randomart image is: 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: +---[RSA 3072]----+ 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | ..+O+=B| 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | ..==B==| 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | o+BB*B| 2024-05-17T04:23:56.827 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | . oo*=O+| 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | S +.+.+.*| 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | o.. oo| 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | . E| 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | | 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: | | 2024-05-17T04:23:56.828 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:23:56 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: +----[SHA256]-----+ 2024-05-17T04:23:57.124 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff716b6700 1 Processor -- start 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff716b6700 1 -- start start 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff716b6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff716b6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c07d780 con 0x7eff6c109e70 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff6affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff6affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55902/0 (socket says 172.21.15.50:55902) 2024-05-17T04:23:57.125 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.667+0000 7eff6affd700 1 -- 172.21.15.50:0/2092990877 learned_addr learned my addr 172.21.15.50:0/2092990877 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.668+0000 7eff6affd700 1 -- 172.21.15.50:0/2092990877 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c10c7d0 con 0x7eff6c109e70 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.668+0000 7eff6affd700 1 --2- 172.21.15.50:0/2092990877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0098a0 tx=0x7eff5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5879955cca95253c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.668+0000 7eff69ffb700 1 -- 172.21.15.50:0/2092990877 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c004160 con 0x7eff6c109e70 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.668+0000 7eff69ffb700 1 -- 172.21.15.50:0/2092990877 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7eff5c004300 con 0x7eff6c109e70 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.668+0000 7eff69ffb700 1 -- 172.21.15.50:0/2092990877 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c0119e0 con 0x7eff6c109e70 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 -- 172.21.15.50:0/2092990877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 msgr2=0x7eff6c10c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 --2- 172.21.15.50:0/2092990877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0098a0 tx=0x7eff5c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 -- 172.21.15.50:0/2092990877 shutdown_connections 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 --2- 172.21.15.50:0/2092990877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c10c290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 -- 172.21.15.50:0/2092990877 >> 172.21.15.50:0/2092990877 conn(0x7eff6c103a40 msgr2=0x7eff6c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 -- 172.21.15.50:0/2092990877 shutdown_connections 2024-05-17T04:23:57.126 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 -- 172.21.15.50:0/2092990877 wait complete. 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.669+0000 7eff716b6700 1 Processor -- start 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff716b6700 1 -- start start 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff716b6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff716b6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c10d6d0 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff6affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff6affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55914/0 (socket says 172.21.15.50:55914) 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff6affd700 1 -- 172.21.15.50:0/1455162633 learned_addr learned my addr 172.21.15.50:0/1455162633 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff6affd700 1 -- 172.21.15.50:0/1455162633 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c19dbb0 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.670+0000 7eff6affd700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0116a0 tx=0x7eff5c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.671+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c003970 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.671+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7eff5c003b10 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.671+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff5c019550 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.671+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c19de40 con 0x7eff6c109e70 2024-05-17T04:23:57.127 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.671+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a0a50 con 0x7eff6c109e70 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.672+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c10a280 con 0x7eff6c109e70 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.672+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7eff5c019810 con 0x7eff6c109e70 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.672+0000 7eff53fff700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 0x7eff54040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.672+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff5c0539d0 con 0x7eff6c109e70 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.672+0000 7eff6a7fc700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 0x7eff54040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.673+0000 7eff6a7fc700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 0x7eff54040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.677+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff5c010380 con 0x7eff6c109e70 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:55.893+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7eff6c104b40 con 0x7eff5403e470 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.565+0000 7eff53fff700 1 -- 172.21.15.50:0/1455162633 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff6c104b40 con 0x7eff5403e470 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 msgr2=0x7eff54040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 0x7eff54040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:57.128 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 msgr2=0x7eff6c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0116a0 tx=0x7eff5c011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 shutdown_connections 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7eff5403e470 0x7eff54040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 --2- 172.21.15.50:0/1455162633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff6c109e70 0x7eff6c19d670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 >> 172.21.15.50:0/1455162633 conn(0x7eff6c103a40 msgr2=0x7eff6c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 shutdown_connections 2024-05-17T04:23:57.129 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:56.567+0000 7eff716b6700 1 -- 172.21.15.50:0/1455162633 wait complete. 2024-05-17T04:23:57.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:57 smithi050 ceph-mon[31739]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:23:57.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:57 smithi050 ceph-mon[31739]: Generating ssh key... 2024-05-17T04:23:57.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:57 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:57.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:57 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.603+0000 7fa0ac336700 1 Processor -- start 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.603+0000 7fa0ac336700 1 -- start start 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0ac336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0ac336700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a407d670 con 0x7fa0a4109e40 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0aa0d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0aa0d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55924/0 (socket says 172.21.15.50:55924) 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0aa0d2700 1 -- 172.21.15.50:0/1194849593 learned_addr learned my addr 172.21.15.50:0/1194849593 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:58.397 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0aa0d2700 1 -- 172.21.15.50:0/1194849593 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a410c7a0 con 0x7fa0a4109e40 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.604+0000 7fa0aa0d2700 1 --2- 172.21.15.50:0/1194849593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098a0 tx=0x7fa0a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3682ca02af2ea915 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.605+0000 7fa0a90d0700 1 -- 172.21.15.50:0/1194849593 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0004160 con 0x7fa0a4109e40 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.605+0000 7fa0a90d0700 1 -- 172.21.15.50:0/1194849593 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa0a0004300 con 0x7fa0a4109e40 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.605+0000 7fa0a90d0700 1 -- 172.21.15.50:0/1194849593 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a00119e0 con 0x7fa0a4109e40 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.605+0000 7fa0ac336700 1 -- 172.21.15.50:0/1194849593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 msgr2=0x7fa0a410c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.605+0000 7fa0ac336700 1 --2- 172.21.15.50:0/1194849593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098a0 tx=0x7fa0a0009bb0 comp rx=0 tx=0).stop 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 -- 172.21.15.50:0/1194849593 shutdown_connections 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 --2- 172.21.15.50:0/1194849593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a410c260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 -- 172.21.15.50:0/1194849593 >> 172.21.15.50:0/1194849593 conn(0x7fa0a4103af0 msgr2=0x7fa0a4105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 -- 172.21.15.50:0/1194849593 shutdown_connections 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 -- 172.21.15.50:0/1194849593 wait complete. 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 Processor -- start 2024-05-17T04:23:58.398 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.606+0000 7fa0ac336700 1 -- start start 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0ac336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0ac336700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a410d6a0 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0aa0d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0aa0d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55934/0 (socket says 172.21.15.50:55934) 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0aa0d2700 1 -- 172.21.15.50:0/3063500002 learned_addr learned my addr 172.21.15.50:0/3063500002 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0aa0d2700 1 -- 172.21.15.50:0/3063500002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a419dba0 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0aa0d2700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00116a0 tx=0x7fa0a0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0003970 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa0a0003b10 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.607+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a419de30 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.608+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0019550 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.608+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a41a0a40 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.608+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a410a250 con 0x7fa0a4109e40 2024-05-17T04:23:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.609+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa0a0019840 con 0x7fa0a4109e40 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.609+0000 7fa0977fe700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 0x7fa0900409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.609+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0a0053ae0 con 0x7fa0a4109e40 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.609+0000 7fa0a98d1700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 0x7fa0900409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.609+0000 7fa0a98d1700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 0x7fa0900409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.614+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0a0016260 con 0x7fa0a4109e40 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.831+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fa0a4105bf0 con 0x7fa09003e4c0 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.831+0000 7fa0977fe700 1 -- 172.21.15.50:0/3063500002 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fa0a4105bf0 con 0x7fa09003e4c0 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.833+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 msgr2=0x7fa0900409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.833+0000 7fa0ac336700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 0x7fa0900409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).stop 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.833+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 msgr2=0x7fa0a419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.833+0000 7fa0ac336700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00116a0 tx=0x7fa0a0011a60 comp rx=0 tx=0).stop 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 shutdown_connections 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fa09003e4c0 0x7fa0900409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:58.400 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 --2- 172.21.15.50:0/3063500002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0a4109e40 0x7fa0a419d660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 >> 172.21.15.50:0/3063500002 conn(0x7fa0a4103af0 msgr2=0x7fa0a41054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 shutdown_connections 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:57.834+0000 7fa0ac336700 1 -- 172.21.15.50:0/3063500002 wait complete. 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:Adding key to root@localhost authorized_keys... 2024-05-17T04:23:58.401 INFO:teuthology.orchestra.run.smithi050.stdout:Adding host smithi050... 2024-05-17T04:23:59.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:58 smithi050 ceph-mon[31739]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:00.000 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:23:59 smithi050 ceph-mon[31739]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:01.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:00 smithi050 ceph-mon[31739]: Deploying cephadm binary to smithi050 2024-05-17T04:24:03.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Added host 'smithi050' with addr '172.21.15.50' 2024-05-17T04:24:03.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.858+0000 7f8c4891a700 1 Processor -- start 2024-05-17T04:24:03.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.858+0000 7f8c4891a700 1 -- start start 2024-05-17T04:24:03.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4891a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:03.931 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4891a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4407d6b0 con 0x7f8c44109e00 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53016/0 (socket says 172.21.15.50:53016) 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4259c700 1 -- 172.21.15.50:0/3179069678 learned_addr learned my addr 172.21.15.50:0/3179069678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4259c700 1 -- 172.21.15.50:0/3179069678 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4410c760 con 0x7f8c44109e00 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.859+0000 7f8c4259c700 1 --2- 172.21.15.50:0/3179069678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098a0 tx=0x7f8c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d265aaa757ba7339 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4159a700 1 -- 172.21.15.50:0/3179069678 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c004160 con 0x7f8c44109e00 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4159a700 1 -- 172.21.15.50:0/3179069678 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c3c004300 con 0x7f8c44109e00 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4159a700 1 -- 172.21.15.50:0/3179069678 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c0119e0 con 0x7f8c44109e00 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 -- 172.21.15.50:0/3179069678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 msgr2=0x7f8c4410c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 --2- 172.21.15.50:0/3179069678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0098a0 tx=0x7f8c3c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:03.932 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 -- 172.21.15.50:0/3179069678 shutdown_connections 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 --2- 172.21.15.50:0/3179069678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c4410c220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 -- 172.21.15.50:0/3179069678 >> 172.21.15.50:0/3179069678 conn(0x7f8c44103af0 msgr2=0x7f8c44105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 -- 172.21.15.50:0/3179069678 shutdown_connections 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.860+0000 7f8c4891a700 1 -- 172.21.15.50:0/3179069678 wait complete. 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.861+0000 7f8c4891a700 1 Processor -- start 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.861+0000 7f8c4891a700 1 -- start start 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4891a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4891a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4410d6c0 con 0x7f8c44109e00 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53028/0 (socket says 172.21.15.50:53028) 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4259c700 1 -- 172.21.15.50:0/1627331774 learned_addr learned my addr 172.21.15.50:0/1627331774 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4259c700 1 -- 172.21.15.50:0/1627331774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c44109ac0 con 0x7f8c44109e00 2024-05-17T04:24:03.933 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4259c700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0116a0 tx=0x7f8c3c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c003970 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c3c003b10 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c44107d10 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.862+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c441081f0 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.863+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c019550 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.863+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8c3c019860 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.864+0000 7f8c337fe700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 0x7f8c2c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.864+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24005360 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.864+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c3c054690 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.864+0000 7f8c41d9b700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 0x7f8c2c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.864+0000 7f8c41d9b700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 0x7f8c2c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:58.869+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c3c020e30 con 0x7f8c44109e00 2024-05-17T04:24:03.934 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:23:59.088+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "addr": "172.21.15.50", "target": ["mon-mgr", ""]}) -- 0x7f8c24000bf0 con 0x7f8c2c03e4c0 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.358+0000 7f8c337fe700 1 -- 172.21.15.50:0/1627331774 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8c24000bf0 con 0x7f8c2c03e4c0 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 msgr2=0x7f8c2c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 0x7f8c2c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 msgr2=0x7f8c44109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0116a0 tx=0x7f8c3c011a60 comp rx=0 tx=0).stop 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 shutdown_connections 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f8c2c03e4c0 0x7f8c2c0409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 --2- 172.21.15.50:0/1627331774 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c44109e00 0x7f8c44109580 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 >> 172.21.15.50:0/1627331774 conn(0x7f8c44103af0 msgr2=0x7f8c441054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 shutdown_connections 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:03.361+0000 7f8c4891a700 1 -- 172.21.15.50:0/1627331774 wait complete. 2024-05-17T04:24:03.935 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mon service... 2024-05-17T04:24:04.497 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:04 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:04.497 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:04 smithi050 ceph-mon[31739]: Added host smithi050 2024-05-17T04:24:04.497 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:04 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcdae63700 1 Processor -- start 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcdae63700 1 -- start start 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcdae63700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcdae63700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efcd407cde0 con 0x7efcd4109e40 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcd8bff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcd8bff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53030/0 (socket says 172.21.15.50:53030) 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.495+0000 7efcd8bff700 1 -- 172.21.15.50:0/1822161643 learned_addr learned my addr 172.21.15.50:0/1822161643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:05.360 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.496+0000 7efcd8bff700 1 -- 172.21.15.50:0/1822161643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd410c7a0 con 0x7efcd4109e40 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.496+0000 7efcd8bff700 1 --2- 172.21.15.50:0/1822161643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efcc80098a0 tx=0x7efcc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d02611e5a89e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.496+0000 7efcd37fe700 1 -- 172.21.15.50:0/1822161643 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc8004160 con 0x7efcd4109e40 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.496+0000 7efcd37fe700 1 -- 172.21.15.50:0/1822161643 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efcc8004300 con 0x7efcd4109e40 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.496+0000 7efcd37fe700 1 -- 172.21.15.50:0/1822161643 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc80119e0 con 0x7efcd4109e40 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 -- 172.21.15.50:0/1822161643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 msgr2=0x7efcd410c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 --2- 172.21.15.50:0/1822161643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efcc80098a0 tx=0x7efcc8009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 -- 172.21.15.50:0/1822161643 shutdown_connections 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 --2- 172.21.15.50:0/1822161643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd410c260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 -- 172.21.15.50:0/1822161643 >> 172.21.15.50:0/1822161643 conn(0x7efcd4103af0 msgr2=0x7efcd4105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 -- 172.21.15.50:0/1822161643 shutdown_connections 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.497+0000 7efcdae63700 1 -- 172.21.15.50:0/1822161643 wait complete. 2024-05-17T04:24:05.361 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcdae63700 1 Processor -- start 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcdae63700 1 -- start start 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcdae63700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcdae63700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efcd410d6a0 con 0x7efcd4109e40 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcd8bff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcd8bff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53042/0 (socket says 172.21.15.50:53042) 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.498+0000 7efcd8bff700 1 -- 172.21.15.50:0/3853429389 learned_addr learned my addr 172.21.15.50:0/3853429389 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcd8bff700 1 -- 172.21.15.50:0/3853429389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd419dba0 con 0x7efcd4109e40 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcd8bff700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efcc80116a0 tx=0x7efcc8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:05.362 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc8003970 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7efcc8003b10 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc8019550 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd419de30 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.499+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41a0a40 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.500+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd410a250 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.500+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7efcc801aa50 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.501+0000 7efcd1ffb700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 0x7efcc40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.501+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efcc8053840 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.501+0000 7efcd3fff700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 0x7efcc40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.501+0000 7efcd3fff700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 0x7efcc40409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.506+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efcc801a430 con 0x7efcd4109e40 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.743+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efcd4105bf0 con 0x7efcc403e4c0 2024-05-17T04:24:05.363 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.748+0000 7efcd1ffb700 1 -- 172.21.15.50:0/3853429389 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efcd4105bf0 con 0x7efcc403e4c0 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 msgr2=0x7efcc40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 0x7efcc40409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 msgr2=0x7efcd419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efcc80116a0 tx=0x7efcc8011a60 comp rx=0 tx=0).stop 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 shutdown_connections 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7efcc403e4c0 0x7efcc40409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 --2- 172.21.15.50:0/3853429389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efcd4109e40 0x7efcd419d660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 >> 172.21.15.50:0/3853429389 conn(0x7efcd4103af0 msgr2=0x7efcd41054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 shutdown_connections 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:04.751+0000 7efcdae63700 1 -- 172.21.15.50:0/3853429389 wait complete. 2024-05-17T04:24:05.364 INFO:teuthology.orchestra.run.smithi050.stdout:Deploying unmanaged mgr service... 2024-05-17T04:24:05.989 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:05 smithi050 ceph-mon[31739]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:05.989 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:05 smithi050 ceph-mon[31739]: Saving service mon spec with placement count:5 2024-05-17T04:24:05.989 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:05 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c14230700 1 Processor -- start 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c14230700 1 -- start start 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c14230700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c14230700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c07d650 con 0x7f9c0c109e60 2024-05-17T04:24:07.063 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c11fcc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c11fcc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53050/0 (socket says 172.21.15.50:53050) 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.818+0000 7f9c11fcc700 1 -- 172.21.15.50:0/2141858341 learned_addr learned my addr 172.21.15.50:0/2141858341 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.819+0000 7f9c11fcc700 1 -- 172.21.15.50:0/2141858341 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c10c7c0 con 0x7f9c0c109e60 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.819+0000 7f9c11fcc700 1 --2- 172.21.15.50:0/2141858341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c080098a0 tx=0x7f9c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68152832e9b16ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.819+0000 7f9c10fca700 1 -- 172.21.15.50:0/2141858341 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c08004160 con 0x7f9c0c109e60 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.819+0000 7f9c10fca700 1 -- 172.21.15.50:0/2141858341 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9c08004300 con 0x7f9c0c109e60 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.819+0000 7f9c10fca700 1 -- 172.21.15.50:0/2141858341 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c080119e0 con 0x7f9c0c109e60 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 -- 172.21.15.50:0/2141858341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 msgr2=0x7f9c0c10c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 --2- 172.21.15.50:0/2141858341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c080098a0 tx=0x7f9c08009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 -- 172.21.15.50:0/2141858341 shutdown_connections 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 --2- 172.21.15.50:0/2141858341 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c10c280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:07.064 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 -- 172.21.15.50:0/2141858341 >> 172.21.15.50:0/2141858341 conn(0x7f9c0c103af0 msgr2=0x7f9c0c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 -- 172.21.15.50:0/2141858341 shutdown_connections 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.820+0000 7f9c14230700 1 -- 172.21.15.50:0/2141858341 wait complete. 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c14230700 1 Processor -- start 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c14230700 1 -- start start 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c14230700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c14230700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c10d6c0 con 0x7f9c0c109e60 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c11fcc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c11fcc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53062/0 (socket says 172.21.15.50:53062) 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c11fcc700 1 -- 172.21.15.50:0/1198697663 learned_addr learned my addr 172.21.15.50:0/1198697663 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c11fcc700 1 -- 172.21.15.50:0/1198697663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c19db90 con 0x7f9c0c109e60 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.821+0000 7f9c11fcc700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c08003f20 tx=0x7f9c08011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.822+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c080038b0 con 0x7f9c0c109e60 2024-05-17T04:24:07.065 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.822+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9c08003a50 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.822+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c08019550 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.822+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c19de20 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.822+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c1a0a30 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.823+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c10a270 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.823+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9c08019800 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.823+0000 7f9c02ffd700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 0x7f9bf8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.823+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9c08053c50 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.823+0000 7f9c117cb700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 0x7f9bf8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.824+0000 7f9c117cb700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 0x7f9bf8040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006fd0 tx=0x7f9bfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:05.828+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c0801a7d0 con 0x7f9c0c109e60 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.060+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9c0c105bd0 con 0x7f9bf803e470 2024-05-17T04:24:07.066 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.065+0000 7f9c02ffd700 1 -- 172.21.15.50:0/1198697663 <== mgr.14120 v2:172.21.15.50:6800/2489127888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9c0c105bd0 con 0x7f9bf803e470 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 msgr2=0x7f9bf8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 0x7f9bf8040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc006fd0 tx=0x7f9bfc006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 msgr2=0x7f9c0c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c08003f20 tx=0x7f9c08011a60 comp rx=0 tx=0).stop 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 shutdown_connections 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f9bf803e470 0x7f9bf8040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 --2- 172.21.15.50:0/1198697663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c0c109e60 0x7f9c0c19d650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 >> 172.21.15.50:0/1198697663 conn(0x7f9c0c103af0 msgr2=0x7f9c0c1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 shutdown_connections 2024-05-17T04:24:07.067 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:06.067+0000 7f9c14230700 1 -- 172.21.15.50:0/1198697663 wait complete. 2024-05-17T04:24:07.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:07 smithi050 ceph-mon[31739]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:07.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:07 smithi050 ceph-mon[31739]: Saving service mgr spec with placement count:2 2024-05-17T04:24:07.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:07 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:08.348 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.499+0000 7f2327e38700 1 Processor -- start 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2327e38700 1 -- start start 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2327e38700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2327e38700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f232007cdc0 con 0x7f2320109e60 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2325bd4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2325bd4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53066/0 (socket says 172.21.15.50:53066) 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2325bd4700 1 -- 172.21.15.50:0/1474982872 learned_addr learned my addr 172.21.15.50:0/1474982872 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.500+0000 7f2325bd4700 1 -- 172.21.15.50:0/1474982872 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232010c7c0 con 0x7f2320109e60 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2325bd4700 1 --2- 172.21.15.50:0/1474982872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7a1c6f41dd1cdf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2324bd2700 1 -- 172.21.15.50:0/1474982872 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c004160 con 0x7f2320109e60 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2324bd2700 1 -- 172.21.15.50:0/1474982872 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f231c004300 con 0x7f2320109e60 2024-05-17T04:24:08.349 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2324bd2700 1 -- 172.21.15.50:0/1474982872 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c0119e0 con 0x7f2320109e60 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2327e38700 1 -- 172.21.15.50:0/1474982872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 msgr2=0x7f232010c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.501+0000 7f2327e38700 1 --2- 172.21.15.50:0/1474982872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 -- 172.21.15.50:0/1474982872 shutdown_connections 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 --2- 172.21.15.50:0/1474982872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232010c280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 -- 172.21.15.50:0/1474982872 >> 172.21.15.50:0/1474982872 conn(0x7f2320103af0 msgr2=0x7f2320105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 -- 172.21.15.50:0/1474982872 shutdown_connections 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 -- 172.21.15.50:0/1474982872 wait complete. 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 Processor -- start 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.502+0000 7f2327e38700 1 -- start start 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2327e38700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2327e38700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f232010d6c0 con 0x7f2320109e60 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2325bd4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:08.350 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2325bd4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53078/0 (socket says 172.21.15.50:53078) 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2325bd4700 1 -- 172.21.15.50:0/1045751281 learned_addr learned my addr 172.21.15.50:0/1045751281 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2325bd4700 1 -- 172.21.15.50:0/1045751281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232019dbd0 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2325bd4700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f231c003f20 tx=0x7f231c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c0038b0 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f231c003a50 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232019de60 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c019550 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.503+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201a0a70 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.504+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f231c019850 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.505+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232010a270 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.505+0000 7f2316ffd700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 0x7f230c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.505+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f231c053bd0 con 0x7f2320109e60 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.505+0000 7f23253d3700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 0x7f230c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:08.351 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.505+0000 7f23253d3700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 0x7f230c040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2310006fd0 tx=0x7f2310006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.510+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f231c017e90 con 0x7f2320109e60 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.694+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 7 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f231c05a080 con 0x7f2320109e60 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.747+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f23201a1510 con 0x7f2320109e60 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.750+0000 7f2316ffd700 1 -- 172.21.15.50:0/1045751281 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f231c023390 con 0x7f2320109e60 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 msgr2=0x7f230c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 0x7f230c040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2310006fd0 tx=0x7f2310006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 msgr2=0x7f232019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f231c003f20 tx=0x7f231c011a60 comp rx=0 tx=0).stop 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 shutdown_connections 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f230c03e420 0x7f230c040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 --2- 172.21.15.50:0/1045751281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2320109e60 0x7f232019d690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 >> 172.21.15.50:0/1045751281 conn(0x7f2320103af0 msgr2=0x7f2320105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 shutdown_connections 2024-05-17T04:24:08.352 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:07.753+0000 7f2327e38700 1 -- 172.21.15.50:0/1045751281 wait complete. 2024-05-17T04:24:08.985 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:08 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:08.985 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:08 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1045751281' entity='client.admin' 2024-05-17T04:24:08.985 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:08 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:09.593 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe949861700 1 Processor -- start 2024-05-17T04:24:09.593 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe949861700 1 -- start start 2024-05-17T04:24:09.593 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe949861700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:09.593 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe949861700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe944108c00 con 0x7fe944108220 2024-05-17T04:24:09.593 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe942ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe942ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37228/0 (socket says 172.21.15.50:37228) 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.749+0000 7fe942ffd700 1 -- 172.21.15.50:0/2851133469 learned_addr learned my addr 172.21.15.50:0/2851133469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.750+0000 7fe942ffd700 1 -- 172.21.15.50:0/2851133469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441093f0 con 0x7fe944108220 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.750+0000 7fe942ffd700 1 --2- 172.21.15.50:0/2851133469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9340098a0 tx=0x7fe934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7a2e8fad278fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.750+0000 7fe941ffb700 1 -- 172.21.15.50:0/2851133469 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe934004160 con 0x7fe944108220 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.750+0000 7fe941ffb700 1 -- 172.21.15.50:0/2851133469 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe934004300 con 0x7fe944108220 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.750+0000 7fe941ffb700 1 -- 172.21.15.50:0/2851133469 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe934011960 con 0x7fe944108220 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 -- 172.21.15.50:0/2851133469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 msgr2=0x7fe944108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 --2- 172.21.15.50:0/2851133469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9340098a0 tx=0x7fe934009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 -- 172.21.15.50:0/2851133469 shutdown_connections 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 --2- 172.21.15.50:0/2851133469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe944108630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 -- 172.21.15.50:0/2851133469 >> 172.21.15.50:0/2851133469 conn(0x7fe944103af0 msgr2=0x7fe944105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:09.594 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 -- 172.21.15.50:0/2851133469 shutdown_connections 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.751+0000 7fe949861700 1 -- 172.21.15.50:0/2851133469 wait complete. 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe949861700 1 Processor -- start 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe949861700 1 -- start start 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe949861700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe949861700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe94410c7a0 con 0x7fe944108220 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe942ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe942ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37244/0 (socket says 172.21.15.50:37244) 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.752+0000 7fe942ffd700 1 -- 172.21.15.50:0/2821285186 learned_addr learned my addr 172.21.15.50:0/2821285186 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:09.595 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe942ffd700 1 -- 172.21.15.50:0/2821285186 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94407d6e0 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe942ffd700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe934011670 tx=0x7fe934011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe934003aa0 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe93401b8d0 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9340197f0 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441a0430 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.753+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441a08b0 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.754+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe934019990 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.754+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe944108630 con 0x7fe944108220 2024-05-17T04:24:09.596 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.755+0000 7fe94885f700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 0x7fe930040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.755+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe934053f00 con 0x7fe944108220 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.755+0000 7fe9427fc700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 0x7fe930040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.755+0000 7fe9427fc700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 0x7fe930040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006fd0 tx=0x7fe92c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.760+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe93401f030 con 0x7fe944108220 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.982+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe9441a0f50 con 0x7fe944108220 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.985+0000 7fe94885f700 1 -- 172.21.15.50:0/2821285186 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe934019d80 con 0x7fe944108220 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 msgr2=0x7fe930040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 0x7fe930040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006fd0 tx=0x7fe92c006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 msgr2=0x7fe94407d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe934011670 tx=0x7fe934011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 shutdown_connections 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fe93003e420 0x7fe930040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:09.597 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 --2- 172.21.15.50:0/2821285186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe944108220 0x7fe94407d1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:09.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.988+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 >> 172.21.15.50:0/2821285186 conn(0x7fe944103af0 msgr2=0x7fe944105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:09.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.989+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 shutdown_connections 2024-05-17T04:24:09.598 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:08.989+0000 7fe949861700 1 -- 172.21.15.50:0/2821285186 wait complete. 2024-05-17T04:24:09.598 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling the dashboard module... 2024-05-17T04:24:10.219 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:09 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2821285186' entity='client.admin' 2024-05-17T04:24:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:10 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2156683167' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-17T04:24:11.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:10 smithi050 ceph-mon[31739]: from='mgr.14120 172.21.15.50:0/2884732173' entity='mgr.x' 2024-05-17T04:24:11.327 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: ignoring --setuser ceph since I am not root 2024-05-17T04:24:11.327 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: ignoring --setgroup ceph since I am not root 2024-05-17T04:24:11.327 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:11.209+0000 7fe482637200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T04:24:11.612 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.000+0000 7fb451ee2700 1 Processor -- start 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.000+0000 7fb451ee2700 1 -- start start 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb451ee2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb451ee2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c108c00 con 0x7fb44c108220 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37258/0 (socket says 172.21.15.50:37258) 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44b7fe700 1 -- 172.21.15.50:0/1511780081 learned_addr learned my addr 172.21.15.50:0/1511780081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44b7fe700 1 -- 172.21.15.50:0/1511780081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c1093f0 con 0x7fb44c108220 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44b7fe700 1 --2- 172.21.15.50:0/1511780081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb60aa4f414f594 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44a7fc700 1 -- 172.21.15.50:0/1511780081 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c004160 con 0x7fb44c108220 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44a7fc700 1 -- 172.21.15.50:0/1511780081 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb43c004300 con 0x7fb44c108220 2024-05-17T04:24:11.613 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.001+0000 7fb44a7fc700 1 -- 172.21.15.50:0/1511780081 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c011960 con 0x7fb44c108220 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 -- 172.21.15.50:0/1511780081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 msgr2=0x7fb44c108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 --2- 172.21.15.50:0/1511780081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 -- 172.21.15.50:0/1511780081 shutdown_connections 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 --2- 172.21.15.50:0/1511780081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c108630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 -- 172.21.15.50:0/1511780081 >> 172.21.15.50:0/1511780081 conn(0x7fb44c103af0 msgr2=0x7fb44c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 -- 172.21.15.50:0/1511780081 shutdown_connections 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.002+0000 7fb451ee2700 1 -- 172.21.15.50:0/1511780081 wait complete. 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb451ee2700 1 Processor -- start 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb451ee2700 1 -- start start 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb451ee2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb451ee2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10c7a0 con 0x7fb44c108220 2024-05-17T04:24:11.614 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb44b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb44b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37272/0 (socket says 172.21.15.50:37272) 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.003+0000 7fb44b7fe700 1 -- 172.21.15.50:0/2156683167 learned_addr learned my addr 172.21.15.50:0/2156683167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb44b7fe700 1 -- 172.21.15.50:0/2156683167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c07ce50 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb44b7fe700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb43c011670 tx=0x7fb43c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c01f070 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb43c003b20 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1a0430 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb43c019580 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.004+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c1a08b0 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.005+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c108630 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.005+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb43c022070 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.006+0000 7fb448ff9700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 0x7fb4340409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.006+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb43c053a20 con 0x7fb44c108220 2024-05-17T04:24:11.615 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.006+0000 7fb44affd700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 0x7fb4340409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.006+0000 7fb44affd700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 0x7fb4340409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb440006fd0 tx=0x7fb440006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.010+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb43c017e90 con 0x7fb44c108220 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.293+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb44c1a1520 con 0x7fb44c108220 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.990+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb43c02cd60 con 0x7fb44c108220 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.992+0000 7fb448ff9700 1 -- 172.21.15.50:0/2156683167 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fb43c019cc0 con 0x7fb44c108220 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 msgr2=0x7fb4340409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 0x7fb4340409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb440006fd0 tx=0x7fb440006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 msgr2=0x7fb44c07c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb43c011670 tx=0x7fb43c011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 shutdown_connections 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7fb43403e4c0 0x7fb4340409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 --2- 172.21.15.50:0/2156683167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb44c108220 0x7fb44c07c910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:11.616 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 >> 172.21.15.50:0/2156683167 conn(0x7fb44c103af0 msgr2=0x7fb44c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:11.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 shutdown_connections 2024-05-17T04:24:11.617 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:10.993+0000 7fb451ee2700 1 -- 172.21.15.50:0/2156683167 wait complete. 2024-05-17T04:24:11.644 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:11.406+0000 7fe482637200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:24:11.644 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:11.514+0000 7fe482637200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T04:24:11.947 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:11.854+0000 7fe482637200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T04:24:11.947 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:11.946+0000 7fe482637200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T04:24:12.293 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:11 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2156683167' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-17T04:24:12.293 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:11 smithi050 ceph-mon[31739]: mgrmap e9: x(active, since 19s) 2024-05-17T04:24:12.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:24:12.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-17T04:24:12.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-17T04:24:12.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-17T04:24:12.996 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.084+0000 7f1b9e834700 1 Processor -- start 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.084+0000 7f1b9e834700 1 -- start start 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b9e834700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b9e834700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98108c00 con 0x7f1b98108220 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37290/0 (socket says 172.21.15.50:37290) 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b97fff700 1 -- 172.21.15.50:0/1196565097 learned_addr learned my addr 172.21.15.50:0/1196565097 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.085+0000 7f1b97fff700 1 -- 172.21.15.50:0/1196565097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b981093f0 con 0x7f1b98108220 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.086+0000 7f1b97fff700 1 --2- 172.21.15.50:0/1196565097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098a0 tx=0x7f1b88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2b79a9cdd4e15e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:12.997 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.086+0000 7f1b96ffd700 1 -- 172.21.15.50:0/1196565097 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b88004160 con 0x7f1b98108220 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.086+0000 7f1b96ffd700 1 -- 172.21.15.50:0/1196565097 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b88004300 con 0x7f1b98108220 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.086+0000 7f1b96ffd700 1 -- 172.21.15.50:0/1196565097 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b88011960 con 0x7f1b98108220 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 -- 172.21.15.50:0/1196565097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 msgr2=0x7f1b98108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 --2- 172.21.15.50:0/1196565097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098a0 tx=0x7f1b88009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 -- 172.21.15.50:0/1196565097 shutdown_connections 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 --2- 172.21.15.50:0/1196565097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b98108630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 -- 172.21.15.50:0/1196565097 >> 172.21.15.50:0/1196565097 conn(0x7f1b98103af0 msgr2=0x7f1b98105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 -- 172.21.15.50:0/1196565097 shutdown_connections 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.087+0000 7f1b9e834700 1 -- 172.21.15.50:0/1196565097 wait complete. 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.088+0000 7f1b9e834700 1 Processor -- start 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.088+0000 7f1b9e834700 1 -- start start 2024-05-17T04:24:12.998 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b9e834700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b9e834700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9810c7a0 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37294/0 (socket says 172.21.15.50:37294) 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b97fff700 1 -- 172.21.15.50:0/2961809933 learned_addr learned my addr 172.21.15.50:0/2961809933 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.089+0000 7f1b97fff700 1 -- 172.21.15.50:0/2961809933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9819cac0 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b97fff700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b88011670 tx=0x7f1b88011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b88003aa0 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b8801b8d0 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b880197f0 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9819cd50 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.090+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9819f960 con 0x7f1b98108220 2024-05-17T04:24:12.999 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.091+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b98108630 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.091+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f1b8801b440 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.092+0000 7f1b957fa700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 0x7f1b84040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.092+0000 7f1b977fe700 1 -- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 msgr2=0x7f1b84040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.092+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b88053f40 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.092+0000 7f1b977fe700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 0x7f1b84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.097+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b8801f030 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.293+0000 7f1b977fe700 1 -- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 msgr2=0x7f1b84040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.293+0000 7f1b977fe700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 0x7f1b84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.408+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1b9819ff50 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.409+0000 7f1b957fa700 1 -- 172.21.15.50:0/2961809933 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1b88017e90 con 0x7f1b98108220 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.415+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 msgr2=0x7f1b84040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:24:13.000 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.415+0000 7f1b9e834700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 0x7f1b84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.415+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 msgr2=0x7f1b9819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.415+0000 7f1b9e834700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b88011670 tx=0x7f1b88011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 shutdown_connections 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f1b8403e470 0x7f1b84040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 --2- 172.21.15.50:0/2961809933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98108220 0x7f1b9819c580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 >> 172.21.15.50:0/2961809933 conn(0x7f1b98103af0 msgr2=0x7f1b98105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 shutdown_connections 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:12.417+0000 7f1b9e834700 1 -- 172.21.15.50:0/2961809933 wait complete. 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for the mgr to restart... 2024-05-17T04:24:13.001 INFO:teuthology.orchestra.run.smithi050.stdout:Waiting for mgr epoch 9... 2024-05-17T04:24:13.250 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:12 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2961809933' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-17T04:24:13.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:13.250+0000 7fe482637200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T04:24:13.577 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:13.439+0000 7fe482637200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T04:24:13.927 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:13.620+0000 7fe482637200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T04:24:13.928 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:13.927+0000 7fe482637200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T04:24:14.240 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:14.025+0000 7fe482637200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T04:24:14.240 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:14.239+0000 7fe482637200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T04:24:14.530 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:14.450+0000 7fe482637200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:24:15.514 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:15.098+0000 7fe482637200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T04:24:15.514 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:15.337+0000 7fe482637200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T04:24:16.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:15.792+0000 7fe482637200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T04:24:16.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:15.896+0000 7fe482637200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T04:24:16.576 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:16.287+0000 7fe482637200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T04:24:17.030 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:16.667+0000 7fe482637200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T04:24:17.030 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:16.767+0000 7fe482637200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T04:24:17.326 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:17.029+0000 7fe482637200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T04:24:17.327 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:17.171+0000 7fe482637200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T04:24:17.327 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:17.275+0000 7fe482637200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T04:24:18.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:17.618+0000 7fe482637200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T04:24:18.077 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:17.717+0000 7fe482637200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T04:24:18.645 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:18 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:18.358+0000 7fe482637200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T04:24:18.645 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:24:18 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:24:18.591+0000 7fe482637200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T04:24:19.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: Active manager daemon x restarted 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: Activating manager daemon x 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: osdmap e3: 0 total, 0 up, 0 in 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: mgrmap e10: x(active, starting, since 0.00372017s) 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: Manager daemon x is now available 2024-05-17T04:24:19.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:19.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-17T04:24:19.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-17T04:24:19.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:19 smithi050 ceph-mon[31739]: mgrmap e11: x(active, since 1.00681s) 2024-05-17T04:24:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout { 2024-05-17T04:24:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-17T04:24:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-17T04:24:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout } 2024-05-17T04:24:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02cdc5b700 1 Processor -- start 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02cdc5b700 1 -- start start 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02cdc5b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02cdc5b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02c807ce20 con 0x7f02c8109e00 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37296/0 (socket says 172.21.15.50:37296) 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.422+0000 7f02c77fe700 1 -- 172.21.15.50:0/2655291254 learned_addr learned my addr 172.21.15.50:0/2655291254 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.423+0000 7f02c77fe700 1 -- 172.21.15.50:0/2655291254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c810c760 con 0x7f02c8109e00 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.423+0000 7f02c77fe700 1 --2- 172.21.15.50:0/2655291254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ea4948f73be216 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.423+0000 7f02c67fc700 1 -- 172.21.15.50:0/2655291254 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0004160 con 0x7f02c8109e00 2024-05-17T04:24:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.423+0000 7f02c67fc700 1 -- 172.21.15.50:0/2655291254 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02b0004300 con 0x7f02c8109e00 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.423+0000 7f02c67fc700 1 -- 172.21.15.50:0/2655291254 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0011960 con 0x7f02c8109e00 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 -- 172.21.15.50:0/2655291254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 msgr2=0x7f02c810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/2655291254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02b00098a0 tx=0x7f02b0009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 -- 172.21.15.50:0/2655291254 shutdown_connections 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/2655291254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c810c220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 -- 172.21.15.50:0/2655291254 >> 172.21.15.50:0/2655291254 conn(0x7f02c8103af0 msgr2=0x7f02c8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 -- 172.21.15.50:0/2655291254 shutdown_connections 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.424+0000 7f02cdc5b700 1 -- 172.21.15.50:0/2655291254 wait complete. 2024-05-17T04:24:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.425+0000 7f02cdc5b700 1 Processor -- start 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.425+0000 7f02cdc5b700 1 -- start start 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02cdc5b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02cdc5b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02c810d700 con 0x7f02c8109e00 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37310/0 (socket says 172.21.15.50:37310) 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c77fe700 1 -- 172.21.15.50:0/357006233 learned_addr learned my addr 172.21.15.50:0/357006233 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c77fe700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c819dbb0 con 0x7f02c8109e00 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c77fe700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02b0011670 tx=0x7f02b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.426+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b0003aa0 con 0x7f02c8109e00 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.427+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c819de40 con 0x7f02c8109e00 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.427+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02b001b8d0 con 0x7f02c8109e00 2024-05-17T04:24:20.194 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.427+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b00197f0 con 0x7f02c8109e00 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.427+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81a0a50 con 0x7f02c8109e00 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f02b0019990 con 0x7f02c8109e00 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c4ff9700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f02b4041090 con 0x7f02b403e470 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f02b0053f50 con 0x7f02c8109e00 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c6ffd700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.428+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.629+0000 7f02c6ffd700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:13.629+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:14.030+0000 7f02c6ffd700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:14.030+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:14.831+0000 7f02c6ffd700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:20.195 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:14.831+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:16.432+0000 7f02c6ffd700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.50:6800/2489127888 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:16.432+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:18.594+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f02b002a350 con 0x7f02c8109e00 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:18.595+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 msgr2=0x7f02b4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:18.595+0000 7f02c4ff9700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.596+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f02b0055bf0 con 0x7f02c8109e00 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.596+0000 7f02c4ff9700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 0x7f02b4043f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.597+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f02b4041090 con 0x7f02b0053ae0 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.597+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 0x7f02b4043f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.598+0000 7f02c6ffd700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 0x7f02b4043f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f02b8003a10 tx=0x7f02b80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.598+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f02b4041090 con 0x7f02b0053ae0 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f02c810a210 con 0x7f02b0053ae0 2024-05-17T04:24:20.196 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02c4ff9700 1 -- 172.21.15.50:0/357006233 <== mgr.14152 v2:172.21.15.50:6800/1662041432 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f02c810a210 con 0x7f02b0053ae0 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 msgr2=0x7f02b4043f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 0x7f02b4043f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f02b8003a10 tx=0x7f02b80092b0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 msgr2=0x7f02c819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02b0011670 tx=0x7f02b0011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 shutdown_connections 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b0053ae0 0x7f02b4043f70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c8109e00 0x7f02c819d670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 --2- 172.21.15.50:0/357006233 >> [v2:172.21.15.50:6800/2489127888,v1:172.21.15.50:6801/2489127888] conn(0x7f02b403e470 0x7f02b4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 >> 172.21.15.50:0/357006233 conn(0x7f02c8103af0 msgr2=0x7f02c8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.607+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 shutdown_connections 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:19.608+0000 7f02cdc5b700 1 -- 172.21.15.50:0/357006233 wait complete. 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:mgr epoch 9 is available 2024-05-17T04:24:20.197 INFO:teuthology.orchestra.run.smithi050.stdout:Generating a dashboard self-signed certificate... 2024-05-17T04:24:20.829 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: [17/May/2024:04:24:19] ENGINE Bus STARTING 2024-05-17T04:24:20.829 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: [17/May/2024:04:24:19] ENGINE Serving on http://172.21.15.50:8765 2024-05-17T04:24:20.829 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-17T04:24:20.829 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-17T04:24:20.829 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: [17/May/2024:04:24:19] ENGINE Serving on https://172.21.15.50:7150 2024-05-17T04:24:20.830 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:20 smithi050 ceph-mon[31739]: [17/May/2024:04:24:19] ENGINE Bus STARTED 2024-05-17T04:24:21.955 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:21 smithi050 ceph-mon[31739]: mgrmap e12: x(active, since 2s) 2024-05-17T04:24:21.955 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:21.955 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:21.955 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:21.962 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.883+0000 7f059e5d7700 1 Processor -- start 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.883+0000 7f059e5d7700 1 -- start start 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f059e5d7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f059e5d7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f059807d670 con 0x7f0598109e40 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f0597fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f0597fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47350/0 (socket says 172.21.15.50:47350) 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f0597fff700 1 -- 172.21.15.50:0/3877031635 learned_addr learned my addr 172.21.15.50:0/3877031635 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f0597fff700 1 -- 172.21.15.50:0/3877031635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059810c7a0 con 0x7f0598109e40 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.884+0000 7f0597fff700 1 --2- 172.21.15.50:0/3877031635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05880098a0 tx=0x7f0588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc709998973fd5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.885+0000 7f0596ffd700 1 -- 172.21.15.50:0/3877031635 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0588004160 con 0x7f0598109e40 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.885+0000 7f0596ffd700 1 -- 172.21.15.50:0/3877031635 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0588004300 con 0x7f0598109e40 2024-05-17T04:24:21.963 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.885+0000 7f0596ffd700 1 -- 172.21.15.50:0/3877031635 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0588011960 con 0x7f0598109e40 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 -- 172.21.15.50:0/3877031635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 msgr2=0x7f059810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 --2- 172.21.15.50:0/3877031635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05880098a0 tx=0x7f0588009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 -- 172.21.15.50:0/3877031635 shutdown_connections 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 --2- 172.21.15.50:0/3877031635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059810c260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 -- 172.21.15.50:0/3877031635 >> 172.21.15.50:0/3877031635 conn(0x7f0598103af0 msgr2=0x7f0598105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 -- 172.21.15.50:0/3877031635 shutdown_connections 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.886+0000 7f059e5d7700 1 -- 172.21.15.50:0/3877031635 wait complete. 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.887+0000 7f059e5d7700 1 Processor -- start 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.887+0000 7f059e5d7700 1 -- start start 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.887+0000 7f059e5d7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f059e5d7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f059810d6e0 con 0x7f0598109e40 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f0597fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:21.964 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f0597fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47362/0 (socket says 172.21.15.50:47362) 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f0597fff700 1 -- 172.21.15.50:0/1723368251 learned_addr learned my addr 172.21.15.50:0/1723368251 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f0597fff700 1 -- 172.21.15.50:0/1723368251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819dbc0 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f0597fff700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0588011670 tx=0x7f0588011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0588003aa0 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819de50 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f058801b8d0 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05880197f0 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.888+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a0a60 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.889+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f0588019990 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.890+0000 7f05957fa700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 0x7f0584040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.890+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0588053ff0 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.890+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059810a250 con 0x7f0598109e40 2024-05-17T04:24:21.965 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.890+0000 7f05977fe700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 0x7f0584040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.890+0000 7f05977fe700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 0x7f0584040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0580006fd0 tx=0x7f0580006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:20.895+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f058801f030 con 0x7f0598109e40 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.141+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0598105bf0 con 0x7f058403e470 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.196+0000 7f05957fa700 1 -- 172.21.15.50:0/1723368251 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0598105bf0 con 0x7f058403e470 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 msgr2=0x7f0584040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 0x7f0584040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0580006fd0 tx=0x7f0580006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 msgr2=0x7f059819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0588011670 tx=0x7f0588011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 shutdown_connections 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f058403e470 0x7f0584040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:21.966 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 --2- 172.21.15.50:0/1723368251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0598109e40 0x7f059819d680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:21.967 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 >> 172.21.15.50:0/1723368251 conn(0x7f0598103af0 msgr2=0x7f05981054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:21.967 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 shutdown_connections 2024-05-17T04:24:21.967 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:21.198+0000 7f059e5d7700 1 -- 172.21.15.50:0/1723368251 wait complete. 2024-05-17T04:24:21.967 INFO:teuthology.orchestra.run.smithi050.stdout:Creating initial admin user... 2024-05-17T04:24:22.901 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:22 smithi050 ceph-mon[31739]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:22.901 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FhXzpIl5.IWc.e0co09VNOUYxcK/otAbIHUgNml3.sfa5F1fT2aKe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715919862, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.436+0000 7f450b4a2700 1 Processor -- start 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.436+0000 7f450b4a2700 1 -- start start 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450b4a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450b4a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f450407cde0 con 0x7f4504109e40 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450923e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450923e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47364/0 (socket says 172.21.15.50:47364) 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450923e700 1 -- 172.21.15.50:0/1198635338 learned_addr learned my addr 172.21.15.50:0/1198635338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:23.917 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450923e700 1 -- 172.21.15.50:0/1198635338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450410c7a0 con 0x7f4504109e40 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f450923e700 1 --2- 172.21.15.50:0/1198635338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45000098a0 tx=0x7f4500009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e2f2c46bd73f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f44fbfff700 1 -- 172.21.15.50:0/1198635338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4500004160 con 0x7f4504109e40 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f44fbfff700 1 -- 172.21.15.50:0/1198635338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4500004300 con 0x7f4504109e40 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.437+0000 7f44fbfff700 1 -- 172.21.15.50:0/1198635338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4500011960 con 0x7f4504109e40 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.438+0000 7f450b4a2700 1 -- 172.21.15.50:0/1198635338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 msgr2=0x7f450410c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.438+0000 7f450b4a2700 1 --2- 172.21.15.50:0/1198635338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45000098a0 tx=0x7f4500009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.438+0000 7f450b4a2700 1 -- 172.21.15.50:0/1198635338 shutdown_connections 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.438+0000 7f450b4a2700 1 --2- 172.21.15.50:0/1198635338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f450410c260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.438+0000 7f450b4a2700 1 -- 172.21.15.50:0/1198635338 >> 172.21.15.50:0/1198635338 conn(0x7f4504103af0 msgr2=0x7f4504105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 -- 172.21.15.50:0/1198635338 shutdown_connections 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 -- 172.21.15.50:0/1198635338 wait complete. 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 Processor -- start 2024-05-17T04:24:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 -- start start 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.439+0000 7f450b4a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f450410d6e0 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450923e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450923e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47378/0 (socket says 172.21.15.50:47378) 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450923e700 1 -- 172.21.15.50:0/498474720 learned_addr learned my addr 172.21.15.50:0/498474720 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450923e700 1 -- 172.21.15.50:0/498474720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4504109b00 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450923e700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4500011670 tx=0x7f4500011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f450001f070 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4500003b20 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4504107d50 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4500019580 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.440+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4504108230 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.441+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450410a250 con 0x7f4504109e40 2024-05-17T04:24:23.919 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.441+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f4500022070 con 0x7f4504109e40 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.442+0000 7f44fa7fc700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 0x7f44f0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.442+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4500053b10 con 0x7f4504109e40 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.442+0000 7f4508a3d700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 0x7f44f0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.442+0000 7f4508a3d700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 0x7f44f0040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.447+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45000172b0 con 0x7f4504109e40 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.682+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4504105c00 con 0x7f44f003e520 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.931+0000 7f44fa7fc700 1 -- 172.21.15.50:0/498474720 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4504105c00 con 0x7f44f003e520 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 msgr2=0x7f44f0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 0x7f44f0040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 msgr2=0x7f45041095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4500011670 tx=0x7f4500011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 shutdown_connections 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44f003e520 0x7f44f0040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:23.920 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 --2- 172.21.15.50:0/498474720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4504109e40 0x7f45041095c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:23.921 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 >> 172.21.15.50:0/498474720 conn(0x7f4504103af0 msgr2=0x7f45041054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:23.921 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 shutdown_connections 2024-05-17T04:24:23.921 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:22.933+0000 7f450b4a2700 1 -- 172.21.15.50:0/498474720 wait complete. 2024-05-17T04:24:23.921 INFO:teuthology.orchestra.run.smithi050.stdout:Fetching dashboard port number... 2024-05-17T04:24:24.211 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:23 smithi050 ceph-mon[31739]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:24.212 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:25.260 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:24 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3746358661' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stdout 8443 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f781a2ec700 1 Processor -- start 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f781a2ec700 1 -- start start 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f781a2ec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f781a2ec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7814109d00 con 0x7f7814109320 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f7813fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:25.268 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f7813fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47390/0 (socket says 172.21.15.50:47390) 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f7813fff700 1 -- 172.21.15.50:0/1026916179 learned_addr learned my addr 172.21.15.50:0/1026916179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f7813fff700 1 -- 172.21.15.50:0/1026916179 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781410a4f0 con 0x7f7814109320 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.328+0000 7f7813fff700 1 --2- 172.21.15.50:0/1026916179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3bde250c03984 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f78137fe700 1 -- 172.21.15.50:0/1026916179 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804004160 con 0x7f7814109320 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f78137fe700 1 -- 172.21.15.50:0/1026916179 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7804004300 con 0x7f7814109320 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f78137fe700 1 -- 172.21.15.50:0/1026916179 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804011960 con 0x7f7814109320 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f781a2ec700 1 -- 172.21.15.50:0/1026916179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 msgr2=0x7f7814109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f781a2ec700 1 --2- 172.21.15.50:0/1026916179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78040098a0 tx=0x7f7804009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f781a2ec700 1 -- 172.21.15.50:0/1026916179 shutdown_connections 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f781a2ec700 1 --2- 172.21.15.50:0/1026916179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f7814109730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.329+0000 7f781a2ec700 1 -- 172.21.15.50:0/1026916179 >> 172.21.15.50:0/1026916179 conn(0x7f7814104bb0 msgr2=0x7f7814107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 -- 172.21.15.50:0/1026916179 shutdown_connections 2024-05-17T04:24:25.269 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 -- 172.21.15.50:0/1026916179 wait complete. 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 Processor -- start 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 -- start start 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.330+0000 7f781a2ec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f781410d8a0 con 0x7f7814109320 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7813fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7813fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47402/0 (socket says 172.21.15.50:47402) 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7813fff700 1 -- 172.21.15.50:0/3746358661 learned_addr learned my addr 172.21.15.50:0/3746358661 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7813fff700 1 -- 172.21.15.50:0/3746358661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781419dbc0 con 0x7f7814109320 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7813fff700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7804011670 tx=0x7f7804011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7804003aa0 con 0x7f7814109320 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781419de50 con 0x7f7814109320 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f780401b8d0 con 0x7f7814109320 2024-05-17T04:24:25.270 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78040197f0 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.331+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a0a60 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.332+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7814109730 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.332+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f7804019990 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.333+0000 7f7811ffb700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 0x7f7800040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.333+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7804054170 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.333+0000 7f780bfff700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 0x7f7800040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.333+0000 7f780bfff700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 0x7f7800040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.338+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f780401f030 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.553+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f78141a1050 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.554+0000 7f7811ffb700 1 -- 172.21.15.50:0/3746358661 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7804019dd0 con 0x7f7814109320 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 msgr2=0x7f7800040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 0x7f7800040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:25.271 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 msgr2=0x7f781419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7804011670 tx=0x7f7804011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 shutdown_connections 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780003e470 0x7f7800040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 --2- 172.21.15.50:0/3746358661 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7814109320 0x7f781419d680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 >> 172.21.15.50:0/3746358661 conn(0x7f7814104bb0 msgr2=0x7f78141054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 shutdown_connections 2024-05-17T04:24:25.272 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:24.557+0000 7f781a2ec700 1 -- 172.21.15.50:0/3746358661 wait complete. 2024-05-17T04:24:25.276 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-17T04:24:25.276 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout disabled 2024-05-17T04:24:25.284 INFO:teuthology.orchestra.run.smithi050.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-17T04:24:25.284 INFO:teuthology.orchestra.run.smithi050.stdout:systemctl: stdout inactive 2024-05-17T04:24:25.284 INFO:teuthology.orchestra.run.smithi050.stdout:firewalld.service is not enabled 2024-05-17T04:24:25.284 INFO:teuthology.orchestra.run.smithi050.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout:Ceph Dashboard is now available at: 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout: URL: https://smithi050.front.sepia.ceph.com:8443/ 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout: User: admin 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout: Password: 9ckz319pgn 2024-05-17T04:24:25.286 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:25.287 INFO:teuthology.orchestra.run.smithi050.stdout:Saving cluster configuration to /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config directory 2024-05-17T04:24:25.287 INFO:teuthology.orchestra.run.smithi050.stdout:Enabling autotune for osd_memory_target 2024-05-17T04:24:26.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:25 smithi050 ceph-mon[31739]: mgrmap e13: x(active, since 6s) 2024-05-17T04:24:26.541 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.737+0000 7f8248929700 1 Processor -- start 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.737+0000 7f8248929700 1 -- start start 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f8248929700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f8248929700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8244108c00 con 0x7f8244108220 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47408/0 (socket says 172.21.15.50:47408) 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824259c700 1 -- 172.21.15.50:0/52278246 learned_addr learned my addr 172.21.15.50:0/52278246 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824259c700 1 -- 172.21.15.50:0/52278246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441093f0 con 0x7f8244108220 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824259c700 1 --2- 172.21.15.50:0/52278246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098a0 tx=0x7f823c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf3a1dff4452a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.738+0000 7f824159a700 1 -- 172.21.15.50:0/52278246 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c004160 con 0x7f8244108220 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f824159a700 1 -- 172.21.15.50:0/52278246 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823c004300 con 0x7f8244108220 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f824159a700 1 -- 172.21.15.50:0/52278246 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c011960 con 0x7f8244108220 2024-05-17T04:24:26.542 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 -- 172.21.15.50:0/52278246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 msgr2=0x7f8244108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 --2- 172.21.15.50:0/52278246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098a0 tx=0x7f823c009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 -- 172.21.15.50:0/52278246 shutdown_connections 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 --2- 172.21.15.50:0/52278246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244108630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 -- 172.21.15.50:0/52278246 >> 172.21.15.50:0/52278246 conn(0x7f8244103af0 msgr2=0x7f8244105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 -- 172.21.15.50:0/52278246 shutdown_connections 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.739+0000 7f8248929700 1 -- 172.21.15.50:0/52278246 wait complete. 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.740+0000 7f8248929700 1 Processor -- start 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.740+0000 7f8248929700 1 -- start start 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.740+0000 7f8248929700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.740+0000 7f8248929700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f824410c7a0 con 0x7f8244108220 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f824259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f824259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47422/0 (socket says 172.21.15.50:47422) 2024-05-17T04:24:26.543 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f824259c700 1 -- 172.21.15.50:0/3009506277 learned_addr learned my addr 172.21.15.50:0/3009506277 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f824259c700 1 -- 172.21.15.50:0/3009506277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8244107270 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f824259c700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f823c011670 tx=0x7f823c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c003aa0 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823c01b8d0 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441054c0 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c0197f0 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.741+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441059a0 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.742+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244108630 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.742+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f823c019990 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.743+0000 7f823b7fe700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 0x7f8230040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.743+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f823c054100 con 0x7f8244108220 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.743+0000 7f8241d9b700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 0x7f8230040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.743+0000 7f8241d9b700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 0x7f8230040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:26.544 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.748+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f823c01f030 con 0x7f8244108220 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.960+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f8244196920 con 0x7f8244108220 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.961+0000 7f823b7fe700 1 -- 172.21.15.50:0/3009506277 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f823c00b3e0 con 0x7f8244108220 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 msgr2=0x7f8230040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 0x7f8230040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 msgr2=0x7f8244106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f823c011670 tx=0x7f823c011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 shutdown_connections 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f823003e510 0x7f8230040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 --2- 172.21.15.50:0/3009506277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8244108220 0x7f8244106d30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.963+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 >> 172.21.15.50:0/3009506277 conn(0x7f8244103af0 msgr2=0x7f8244193c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.964+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 shutdown_connections 2024-05-17T04:24:26.545 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:25.964+0000 7f8248929700 1 -- 172.21.15.50:0/3009506277 wait complete. 2024-05-17T04:24:27.882 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.945+0000 7fd3a3428700 1 Processor -- start 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.945+0000 7fd3a3428700 1 -- start start 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.945+0000 7fd3a3428700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.945+0000 7fd3a3428700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c108c10 con 0x7fd39c108230 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd3a11c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd3a11c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47438/0 (socket says 172.21.15.50:47438) 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd3a11c4700 1 -- 172.21.15.50:0/994522300 learned_addr learned my addr 172.21.15.50:0/994522300 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd3a11c4700 1 -- 172.21.15.50:0/994522300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c109400 con 0x7fd39c108230 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd3a11c4700 1 --2- 172.21.15.50:0/994522300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd3980098a0 tx=0x7fd398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bc8282bf471ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd393fff700 1 -- 172.21.15.50:0/994522300 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398004160 con 0x7fd39c108230 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd393fff700 1 -- 172.21.15.50:0/994522300 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd398004300 con 0x7fd39c108230 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.946+0000 7fd393fff700 1 -- 172.21.15.50:0/994522300 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398011960 con 0x7fd39c108230 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 -- 172.21.15.50:0/994522300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 msgr2=0x7fd39c108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:27.883 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 --2- 172.21.15.50:0/994522300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd3980098a0 tx=0x7fd398009bb0 comp rx=0 tx=0).stop 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 -- 172.21.15.50:0/994522300 shutdown_connections 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 --2- 172.21.15.50:0/994522300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c108640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 -- 172.21.15.50:0/994522300 >> 172.21.15.50:0/994522300 conn(0x7fd39c103a40 msgr2=0x7fd39c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 -- 172.21.15.50:0/994522300 shutdown_connections 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.947+0000 7fd3a3428700 1 -- 172.21.15.50:0/994522300 wait complete. 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a3428700 1 Processor -- start 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a3428700 1 -- start start 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a3428700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a3428700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c7b0 con 0x7fd39c108230 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a11c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a11c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47446/0 (socket says 172.21.15.50:47446) 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.948+0000 7fd3a11c4700 1 -- 172.21.15.50:0/785103431 learned_addr learned my addr 172.21.15.50:0/785103431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:27.884 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3a11c4700 1 -- 172.21.15.50:0/785103431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c07d810 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3a11c4700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd398011670 tx=0x7fd398011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd398003aa0 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd39801b8d0 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c1a0400 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3980197f0 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.949+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c1a0880 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.950+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd398019990 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.950+0000 7fd3927fc700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 0x7fd388040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.950+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c108640 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.951+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3980543f0 con 0x7fd39c108230 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.951+0000 7fd3a09c3700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 0x7fd388040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.951+0000 7fd3a09c3700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 0x7fd388040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd38c006fd0 tx=0x7fd38c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:27.885 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:26.956+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd39801f030 con 0x7fd39c108230 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.294+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd39c1a0f20 con 0x7fd39c108230 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.296+0000 7fd3927fc700 1 -- 172.21.15.50:0/785103431 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fd39800b3e0 con 0x7fd39c108230 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 msgr2=0x7fd388040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 0x7fd388040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd38c006fd0 tx=0x7fd38c006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 msgr2=0x7fd39c07d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd398011670 tx=0x7fd398011a30 comp rx=0 tx=0).stop 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 shutdown_connections 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd38803e570 0x7fd388040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 --2- 172.21.15.50:0/785103431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd39c108230 0x7fd39c07d2d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 >> 172.21.15.50:0/785103431 conn(0x7fd39c103a40 msgr2=0x7fd39c1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 shutdown_connections 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr 2024-05-17T04:24:27.299+0000 7fd3a3428700 1 -- 172.21.15.50:0/785103431 wait complete. 2024-05-17T04:24:27.886 INFO:teuthology.orchestra.run.smithi050.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout:Or, if you are only running a single cluster on this host: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: ceph telemetry on 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout:For more information see: 2024-05-17T04:24:27.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.888 INFO:teuthology.orchestra.run.smithi050.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-17T04:24:27.888 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:27.888 INFO:teuthology.orchestra.run.smithi050.stdout:Bootstrap complete. 2024-05-17T04:24:27.929 INFO:tasks.cephadm:Fetching config... 2024-05-17T04:24:27.929 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:24:27.929 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-17T04:24:27.947 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-17T04:24:27.947 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:24:27.948 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-17T04:24:28.006 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-17T04:24:28.006 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:24:28.006 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-17T04:24:28.073 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-17T04:24:28.073 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:24:28.073 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-17T04:24:28.130 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-17T04:24:28.131 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T04:24:28.208 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:24:28.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T04:24:28.269 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:24:28.282 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-17T04:24:28.322 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+1Y4wevYJrujX34R5NWxba4DbLiyCnxbVE/v2yGXoMCfP/U6D5Vq1Ycny0NL+cyCItsPWGpAhY/yxp92xpwKe+k5NtxX/1Sqg9RtmieuR5aAVWQSfCuprX5Ao/yUu0kxOAURg6Ys2fQ7FMb2hqzes7dyznwNRYGCjsFzWL4nmGJeQE/FIq56RZIdWCOtTeHwqi/GgVZMboMPgrZ+CvS7gFSFjdDSJBOTxptV94pY05ae928o70KNo8wJsdAEUGlCjwwyDDUJfpJ3pOYS4iTwSL5bz0ybNIUGNT/s6ZExnBkFkd3pIK+NnE+fQwhZIjNuuJvwdaw4MGzYDzuK6Z5PZq9LCsA1hIEFcr0tDsrCqEH2ZwmAvxuyIoRAPckgDwB2G1S60UAUUVCtLmyZGHYIm5snv6Rl8r2d9R/+WvWWvpZhMhJCcuGwyui8OLfIB3TNd44hoZXLMCK2WlBzKZG6TQ/jScptS0dD+EbhtLqrrv8eep4nr4OcBzYwto2x67IU= ceph-0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:24:28.335 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-17T04:24:28.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:28 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/785103431' entity='client.admin' 2024-05-17T04:24:28.579 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 -- 172.21.15.50:0/1647083175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 msgr2=0x7f87e0102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:30.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 --2- 172.21.15.50:0/1647083175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0102860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b0a0 tx=0x7f87d000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 -- 172.21.15.50:0/1647083175 shutdown_connections 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 --2- 172.21.15.50:0/1647083175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0102860 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 -- 172.21.15.50:0/1647083175 >> 172.21.15.50:0/1647083175 conn(0x7f87e00fdc80 msgr2=0x7f87e0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 -- 172.21.15.50:0/1647083175 shutdown_connections 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.753+0000 7f87e58a3700 1 -- 172.21.15.50:0/1647083175 wait complete. 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.754+0000 7f87e58a3700 1 Processor -- start 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.754+0000 7f87e58a3700 1 -- start start 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.754+0000 7f87e58a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.754+0000 7f87e58a3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01082d0 con 0x7f87e0102450 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87deffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87deffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47316/0 (socket says 172.21.15.50:47316) 2024-05-17T04:24:30.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87deffd700 1 -- 172.21.15.50:0/3375500873 learned_addr learned my addr 172.21.15.50:0/3375500873 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87deffd700 1 -- 172.21.15.50:0/3375500873 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e0199530 con 0x7f87e0102450 2024-05-17T04:24:30.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87deffd700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87d0015040 tx=0x7f87d000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d0017040 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e01997c0 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87d0022070 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87d00125c0 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.755+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e019c3d0 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.756+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f87d00128c0 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.756+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e0108a50 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.757+0000 7f87e48a1700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 0x7f87c8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.757+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f87d000f050 con 0x7f87e0102450 2024-05-17T04:24:30.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.757+0000 7f87de7fc700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 0x7f87c8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:30.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.757+0000 7f87de7fc700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 0x7f87c8040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:30.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.762+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87d001a350 con 0x7f87e0102450 2024-05-17T04:24:30.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.975+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f87e0064440 con 0x7f87e0102450 2024-05-17T04:24:30.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.977+0000 7f87e48a1700 1 -- 172.21.15.50:0/3375500873 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f87d002d250 con 0x7f87e0102450 2024-05-17T04:24:30.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.979+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 msgr2=0x7f87c8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:30.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.979+0000 7f87e58a3700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 0x7f87c8040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f87d4006fd0 tx=0x7f87d4006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:30.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.979+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 msgr2=0x7f87e0198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:30.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.979+0000 7f87e58a3700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87d0015040 tx=0x7f87d000bd40 comp rx=0 tx=0).stop 2024-05-17T04:24:30.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 shutdown_connections 2024-05-17T04:24:30.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f87c803e510 0x7f87c8040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:30.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 --2- 172.21.15.50:0/3375500873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f87e0102450 0x7f87e0198ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:30.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 >> 172.21.15.50:0/3375500873 conn(0x7f87e00fdc80 msgr2=0x7f87e006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:30.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 shutdown_connections 2024-05-17T04:24:30.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:30.980+0000 7f87e58a3700 1 -- 172.21.15.50:0/3375500873 wait complete. 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:31.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:31.611 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-17T04:24:31.611 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-17T04:24:31.856 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:32.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:31 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3375500873' entity='client.admin' 2024-05-17T04:24:32.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:32.283 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.372+0000 7f596cf52700 1 -- 172.21.15.50:0/1802556956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 msgr2=0x7f59681064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.372+0000 7f596cf52700 1 --2- 172.21.15.50:0/1802556956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f59681064a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595800b0a0 tx=0x7f595800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.372+0000 7f596cf52700 1 -- 172.21.15.50:0/1802556956 shutdown_connections 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.372+0000 7f596cf52700 1 --2- 172.21.15.50:0/1802556956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f59681064a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.372+0000 7f596cf52700 1 -- 172.21.15.50:0/1802556956 >> 172.21.15.50:0/1802556956 conn(0x7f59680fdcb0 msgr2=0x7f5968100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.373+0000 7f596cf52700 1 -- 172.21.15.50:0/1802556956 shutdown_connections 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.373+0000 7f596cf52700 1 -- 172.21.15.50:0/1802556956 wait complete. 2024-05-17T04:24:33.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.373+0000 7f596cf52700 1 Processor -- start 2024-05-17T04:24:33.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.373+0000 7f596cf52700 1 -- start start 2024-05-17T04:24:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596cf52700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:33.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596cf52700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f59681093f0 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47342/0 (socket says 172.21.15.50:47342) 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596659c700 1 -- 172.21.15.50:0/1577757324 learned_addr learned my addr 172.21.15.50:0/1577757324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596659c700 1 -- 172.21.15.50:0/1577757324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5968103d40 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f596659c700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f595800bc90 tx=0x7f595800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5958017040 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.374+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5958022070 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.375+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5968101f90 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.375+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59580048f0 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.375+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5968102470 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5968109b70 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5958027460 con 0x7f5968104080 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f595f7fe700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 0x7f5954040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:33.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f595800f050 con 0x7f5968104080 2024-05-17T04:24:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f5965d9b700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 0x7f5954040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:33.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.376+0000 7f5965d9b700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 0x7f5954040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5950006fd0 tx=0x7f5950006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:33.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.381+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5958025e40 con 0x7f5968104080 2024-05-17T04:24:33.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.598+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5968192180 con 0x7f595403e570 2024-05-17T04:24:33.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.601+0000 7f595f7fe700 1 -- 172.21.15.50:0/1577757324 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5968192180 con 0x7f595403e570 2024-05-17T04:24:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 msgr2=0x7f5954040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 0x7f5954040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5950006fd0 tx=0x7f5950006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:33.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 msgr2=0x7f5968103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f595800bc90 tx=0x7f595800bd70 comp rx=0 tx=0).stop 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 shutdown_connections 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f595403e570 0x7f5954040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 --2- 172.21.15.50:0/1577757324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5968104080 0x7f5968103800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 >> 172.21.15.50:0/1577757324 conn(0x7f59680fdcb0 msgr2=0x7f59680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 shutdown_connections 2024-05-17T04:24:33.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:33.604+0000 7f596cf52700 1 -- 172.21.15.50:0/1577757324 wait complete. 2024-05-17T04:24:34.367 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi105 2024-05-17T04:24:34.367 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:24:34.367 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.conf 2024-05-17T04:24:34.389 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:24:34.389 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:34.447 INFO:tasks.cephadm:Adding host smithi105 to orchestrator... 2024-05-17T04:24:34.447 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch host add smithi105 2024-05-17T04:24:34.699 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:34.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:34.923 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:24:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:24:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-17T04:24:34.924 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:34 smithi050 ceph-mon[31739]: Updating smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:24:35.728 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:35 smithi050 ceph-mon[31739]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:35.728 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:35 smithi050 ceph-mon[31739]: Updating smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T04:24:35.728 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:35.729 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:35.729 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 -- 172.21.15.50:0/3152175525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 msgr2=0x7ff770104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 --2- 172.21.15.50:0/3152175525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770104a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff760009910 tx=0x7ff760009c20 comp rx=0 tx=0).stop 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 -- 172.21.15.50:0/3152175525 shutdown_connections 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 --2- 172.21.15.50:0/3152175525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770104a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 -- 172.21.15.50:0/3152175525 >> 172.21.15.50:0/3152175525 conn(0x7ff7700fe160 msgr2=0x7ff7701005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 -- 172.21.15.50:0/3152175525 shutdown_connections 2024-05-17T04:24:36.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.215+0000 7ff775bbb700 1 -- 172.21.15.50:0/3152175525 wait complete. 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff775bbb700 1 Processor -- start 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff775bbb700 1 -- start start 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff775bbb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff775bbb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff77010d970 con 0x7ff770104660 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47366/0 (socket says 172.21.15.50:47366) 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.216+0000 7ff76f7fe700 1 -- 172.21.15.50:0/4018509854 learned_addr learned my addr 172.21.15.50:0/4018509854 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff76f7fe700 1 -- 172.21.15.50:0/4018509854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7701997c0 con 0x7ff770104660 2024-05-17T04:24:36.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff76f7fe700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff760005060 tx=0x7ff760005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:36.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff760026070 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff770199a50 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff76000b750 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff760003850 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.217+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff770199eb0 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.218+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff760029070 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.218+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77006c0a0 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.218+0000 7ff76cff9700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 0x7ff758040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.218+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff76001e070 con 0x7ff770104660 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.218+0000 7ff76effd700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 0x7ff758040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:36.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.219+0000 7ff76effd700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 0x7ff758040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff764006fd0 tx=0x7ff764006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:36.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.223+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff760013070 con 0x7ff770104660 2024-05-17T04:24:36.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:36.443+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}) -- 0x7ff77019a650 con 0x7ff75803e510 2024-05-17T04:24:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:37 smithi050 ceph-mon[31739]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:37 smithi050 ceph-mon[31739]: Deploying cephadm binary to smithi105 2024-05-17T04:24:39.882 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:39 smithi050 ceph-mon[31739]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:40.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.628+0000 7ff76cff9700 1 -- 172.21.15.50:0/4018509854 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff77019a650 con 0x7ff75803e510 2024-05-17T04:24:40.631 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi105' with addr '172.21.15.105' 2024-05-17T04:24:40.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 msgr2=0x7ff758040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 0x7ff758040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff764006fd0 tx=0x7ff764006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 msgr2=0x7ff770199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff760005060 tx=0x7ff760005710 comp rx=0 tx=0).stop 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 shutdown_connections 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff75803e510 0x7ff758040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 --2- 172.21.15.50:0/4018509854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff770104660 0x7ff770199280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 >> 172.21.15.50:0/4018509854 conn(0x7ff7700fe160 msgr2=0x7ff7701054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 shutdown_connections 2024-05-17T04:24:40.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:40.631+0000 7ff775bbb700 1 -- 172.21.15.50:0/4018509854 wait complete. 2024-05-17T04:24:41.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T04:24:41.415 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:41.805 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:41 smithi050 ceph-mon[31739]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:41.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:41.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:41 smithi050 ceph-mon[31739]: Added host smithi105 2024-05-17T04:24:41.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:41.806 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 -- 172.21.15.50:0/1653066015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 msgr2=0x7f147c109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 --2- 172.21.15.50:0/1653066015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c109960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1464009910 tx=0x7f1464009c20 comp rx=0 tx=0).stop 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 -- 172.21.15.50:0/1653066015 shutdown_connections 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 --2- 172.21.15.50:0/1653066015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c109960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 -- 172.21.15.50:0/1653066015 >> 172.21.15.50:0/1653066015 conn(0x7f147c078b70 msgr2=0x7f147c078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 -- 172.21.15.50:0/1653066015 shutdown_connections 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.923+0000 7f1483104700 1 -- 172.21.15.50:0/1653066015 wait complete. 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1483104700 1 Processor -- start 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1483104700 1 -- start start 2024-05-17T04:24:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1483104700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1483104700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f147c10d930 con 0x7f147c109550 2024-05-17T04:24:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1480ea0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1480ea0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45298/0 (socket says 172.21.15.50:45298) 2024-05-17T04:24:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1480ea0700 1 -- 172.21.15.50:0/3744827337 learned_addr learned my addr 172.21.15.50:0/3744827337 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:42.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.924+0000 7f1480ea0700 1 -- 172.21.15.50:0/3744827337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147c104930 con 0x7f147c109550 2024-05-17T04:24:42.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1480ea0700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f146400b5c0 tx=0x7f1464005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464026070 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f146400b750 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464003810 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147c103020 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.925+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147c103420 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.926+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147c1099e0 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.926+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1464029070 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.926+0000 7f1479ffb700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 0x7f1468040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.926+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f146401e070 con 0x7f147c109550 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.926+0000 7f147bfff700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 0x7f1468040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:42.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.927+0000 7f147bfff700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 0x7f1468040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f146c00ad30 tx=0x7f146c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:42.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:42.931+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1464013070 con 0x7f147c109550 2024-05-17T04:24:43.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:43.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.152+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f147c0ff490 con 0x7f146803e570 2024-05-17T04:24:43.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.153+0000 7f1479ffb700 1 -- 172.21.15.50:0/3744827337 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f147c0ff490 con 0x7f146803e570 2024-05-17T04:24:43.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:43.155 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}] 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 msgr2=0x7f1468040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 0x7f1468040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f146c00ad30 tx=0x7f146c0093f0 comp rx=0 tx=0).stop 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 msgr2=0x7f147c102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f146400b5c0 tx=0x7f1464005710 comp rx=0 tx=0).stop 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 shutdown_connections 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f146803e570 0x7f1468040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 --2- 172.21.15.50:0/3744827337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f147c109550 0x7f147c102ae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 >> 172.21.15.50:0/3744827337 conn(0x7f147c078b70 msgr2=0x7f147c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:43.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.155+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 shutdown_connections 2024-05-17T04:24:43.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:43.156+0000 7f1483104700 1 -- 172.21.15.50:0/3744827337 wait complete. 2024-05-17T04:24:43.739 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-05-17T04:24:43.739 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:24:43.739 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-05-17T04:24:43.758 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:24:43.758 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:43.816 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-05-17T04:24:43.816 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch host add smithi110 2024-05-17T04:24:44.062 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:43 smithi050 ceph-mon[31739]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:45.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:44 smithi050 ceph-mon[31739]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:24:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 -- 172.21.15.50:0/155584637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 msgr2=0x7fc618102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 --2- 172.21.15.50:0/155584637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc618102a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc608009910 tx=0x7fc608009c20 comp rx=0 tx=0).stop 2024-05-17T04:24:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 -- 172.21.15.50:0/155584637 shutdown_connections 2024-05-17T04:24:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 --2- 172.21.15.50:0/155584637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc618102a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:45.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 -- 172.21.15.50:0/155584637 >> 172.21.15.50:0/155584637 conn(0x7fc6180fe0a0 msgr2=0x7fc6181004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 -- 172.21.15.50:0/155584637 shutdown_connections 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.523+0000 7fc61d04c700 1 -- 172.21.15.50:0/155584637 wait complete. 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.524+0000 7fc61d04c700 1 Processor -- start 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.524+0000 7fc61d04c700 1 -- start start 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.524+0000 7fc61d04c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.524+0000 7fc61d04c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc61810c7b0 con 0x7fc618102610 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc616d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc616d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45316/0 (socket says 172.21.15.50:45316) 2024-05-17T04:24:45.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc616d9d700 1 -- 172.21.15.50:0/4275544762 learned_addr learned my addr 172.21.15.50:0/4275544762 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc616d9d700 1 -- 172.21.15.50:0/4275544762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618199730 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc616d9d700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc608006010 tx=0x7fc608005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608026070 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc618199a20 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc60800b6f0 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc608003810 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.525+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61819da20 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.526+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618103ac0 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.526+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc608029070 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.527+0000 7fc5fffff700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 0x7fc600040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.527+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc60801e070 con 0x7fc618102610 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.527+0000 7fc61659c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 0x7fc600040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:45.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.527+0000 7fc61659c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 0x7fc600040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:45.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.532+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc608013070 con 0x7fc618102610 2024-05-17T04:24:45.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:45.753+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}) -- 0x7fc61819a550 con 0x7fc60003e510 2024-05-17T04:24:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:45 smithi050 ceph-mon[31739]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:47.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:47.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:47.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:24:47.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:24:47.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:46 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:24:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:47 smithi050 ceph-mon[31739]: Deploying cephadm binary to smithi110 2024-05-17T04:24:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:47 smithi050 ceph-mon[31739]: Updating smithi105:/etc/ceph/ceph.conf 2024-05-17T04:24:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:47 smithi050 ceph-mon[31739]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:48.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:47 smithi050 ceph-mon[31739]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:24:48.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:47 smithi050 ceph-mon[31739]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:50.004 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:49 smithi050 ceph-mon[31739]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:50.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.003+0000 7fc5fffff700 1 -- 172.21.15.50:0/4275544762 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc61819a550 con 0x7fc60003e510 2024-05-17T04:24:50.005 INFO:teuthology.orchestra.run.smithi050.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 msgr2=0x7fc600040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 0x7fc600040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 msgr2=0x7fc6181991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc608006010 tx=0x7fc608005710 comp rx=0 tx=0).stop 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 shutdown_connections 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc60003e510 0x7fc600040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 --2- 172.21.15.50:0/4275544762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc618102610 0x7fc6181991f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 >> 172.21.15.50:0/4275544762 conn(0x7fc6180fe0a0 msgr2=0x7fc61806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:50.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 shutdown_connections 2024-05-17T04:24:50.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:50.005+0000 7fc61d04c700 1 -- 172.21.15.50:0/4275544762 wait complete. 2024-05-17T04:24:50.555 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-17T04:24:50.803 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:50.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T04:24:50.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:50.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:50.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:50.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:50.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: Added host smithi110 2024-05-17T04:24:50.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:50.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:52.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:52 smithi050 ceph-mon[31739]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:52.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- 172.21.15.50:0/1078688609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 msgr2=0x7f7820102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 --2- 172.21.15.50:0/1078688609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820102a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7810009910 tx=0x7f7810009c20 comp rx=0 tx=0).stop 2024-05-17T04:24:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- 172.21.15.50:0/1078688609 shutdown_connections 2024-05-17T04:24:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 --2- 172.21.15.50:0/1078688609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820102a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- 172.21.15.50:0/1078688609 >> 172.21.15.50:0/1078688609 conn(0x7f78200fe0b0 msgr2=0x7f7820100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- 172.21.15.50:0/1078688609 shutdown_connections 2024-05-17T04:24:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- 172.21.15.50:0/1078688609 wait complete. 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 Processor -- start 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.341+0000 7f7825a37700 1 -- start start 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f7825a37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f7825a37700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f782010c7d0 con 0x7f7820102630 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f781effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f781effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41760/0 (socket says 172.21.15.50:41760) 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f781effd700 1 -- 172.21.15.50:0/2568287146 learned_addr learned my addr 172.21.15.50:0/2568287146 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:52.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f781effd700 1 -- 172.21.15.50:0/2568287146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78201996d0 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f781effd700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7810005660 tx=0x7f7810005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7810026070 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.342+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f781000b6f0 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.343+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78100037b0 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.343+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78201999c0 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.343+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782019d9c0 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7810029070 con 0x7f7820102630 2024-05-17T04:24:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820103ae0 con 0x7f7820102630 2024-05-17T04:24:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f7824a35700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 0x7f780c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f781001e070 con 0x7f7820102630 2024-05-17T04:24:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f781e7fc700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 0x7f780c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.344+0000 7f781e7fc700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 0x7f780c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:52.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.349+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7810013070 con 0x7f7820102630 2024-05-17T04:24:52.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.569+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f782019a4f0 con 0x7f780c03e510 2024-05-17T04:24:52.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.570+0000 7f7824a35700 1 -- 172.21.15.50:0/2568287146 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f782019a4f0 con 0x7f780c03e510 2024-05-17T04:24:52.571 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:24:52.571 INFO:teuthology.orchestra.run.smithi050.stdout:[{"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-05-17T04:24:52.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 msgr2=0x7f780c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:52.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 0x7f780c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 msgr2=0x7f7820199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7810005660 tx=0x7f7810005740 comp rx=0 tx=0).stop 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 shutdown_connections 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c03e510 0x7f780c040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 --2- 172.21.15.50:0/2568287146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7820102630 0x7f7820199190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 >> 172.21.15.50:0/2568287146 conn(0x7f78200fe0b0 msgr2=0x7f782006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 shutdown_connections 2024-05-17T04:24:52.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:52.573+0000 7f7825a37700 1 -- 172.21.15.50:0/2568287146 wait complete. 2024-05-17T04:24:53.207 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-17T04:24:53.207 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-17T04:24:53.452 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:24:53.830 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:53 smithi050 ceph-mon[31739]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:24:53.830 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:53.830 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:53 smithi050 ceph-mon[31739]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 -- 172.21.15.50:0/1560470621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 msgr2=0x7fc8fc102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 --2- 172.21.15.50:0/1560470621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc102080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009910 tx=0x7fc8ec009c20 comp rx=0 tx=0).stop 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 -- 172.21.15.50:0/1560470621 shutdown_connections 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 --2- 172.21.15.50:0/1560470621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc102080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 -- 172.21.15.50:0/1560470621 >> 172.21.15.50:0/1560470621 conn(0x7fc8fc078b70 msgr2=0x7fc8fc078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 -- 172.21.15.50:0/1560470621 shutdown_connections 2024-05-17T04:24:54.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.845+0000 7fc900e71700 1 -- 172.21.15.50:0/1560470621 wait complete. 2024-05-17T04:24:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.846+0000 7fc900e71700 1 Processor -- start 2024-05-17T04:24:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.846+0000 7fc900e71700 1 -- start start 2024-05-17T04:24:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc900e71700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:54.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc900e71700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc10d880 con 0x7fc8fc101c70 2024-05-17T04:24:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8fa59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8fa59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41788/0 (socket says 172.21.15.50:41788) 2024-05-17T04:24:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8fa59c700 1 -- 172.21.15.50:0/2608592261 learned_addr learned my addr 172.21.15.50:0/2608592261 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:24:54.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8fa59c700 1 -- 172.21.15.50:0/2608592261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc199630 con 0x7fc8fc101c70 2024-05-17T04:24:54.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8fa59c700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005660 tx=0x7fc8ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec026070 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc199920 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8ec00b750 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.847+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ec003810 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.848+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc19d920 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.848+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc104d00 con 0x7fc8fc101c70 2024-05-17T04:24:54.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.849+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc8ec029070 con 0x7fc8fc101c70 2024-05-17T04:24:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.849+0000 7fc8f37fe700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 0x7fc8e8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.849+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8ec01e070 con 0x7fc8fc101c70 2024-05-17T04:24:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.849+0000 7fc8f3fff700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 0x7fc8e8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:54.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.849+0000 7fc8f3fff700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 0x7fc8e8040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:54.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:54.854+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8ec013070 con 0x7fc8fc101c70 2024-05-17T04:24:55.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.078+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc8fc19a450 con 0x7fc8fc101c70 2024-05-17T04:24:55.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.672+0000 7fc8f37fe700 1 -- 172.21.15.50:0/2608592261 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc8ec00e3e0 con 0x7fc8fc101c70 2024-05-17T04:24:55.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 msgr2=0x7fc8e8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:55.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 0x7fc8e8040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4006fd0 tx=0x7fc8e4006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 msgr2=0x7fc8fc1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005660 tx=0x7fc8ec005740 comp rx=0 tx=0).stop 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 shutdown_connections 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc8e803e570 0x7fc8e8040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.676+0000 7fc900e71700 1 --2- 172.21.15.50:0/2608592261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc8fc101c70 0x7fc8fc1990f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.677+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 >> 172.21.15.50:0/2608592261 conn(0x7fc8fc078b70 msgr2=0x7fc8fc106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.677+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 shutdown_connections 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:24:55.677+0000 7fc900e71700 1 -- 172.21.15.50:0/2608592261 wait complete. 2024-05-17T04:24:55.678 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2608592261' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:56.015 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:24:56.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:24:56.016 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:24:56.337 INFO:tasks.cephadm:Adding mon.a on smithi050 2024-05-17T04:24:56.337 INFO:tasks.cephadm:Adding mon.b on smithi105 2024-05-17T04:24:56.337 INFO:tasks.cephadm:Adding mon.c on smithi110 2024-05-17T04:24:56.337 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c' 2024-05-17T04:24:56.571 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:24:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:56 smithi050 ceph-mon[31739]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-17T04:24:57.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:56 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2608592261' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T04:24:57.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:56 smithi050 ceph-mon[31739]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:24:57.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:56 smithi050 ceph-mon[31739]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:24:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:24:58.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T04:24:58.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:24:58.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:58.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:58.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:58.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 -- 172.21.15.110:0/3658503643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 msgr2=0x7f2214102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:58.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 --2- 172.21.15.110:0/3658503643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f2214102a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f220400b0a0 tx=0x7f220400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:24:58.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 -- 172.21.15.110:0/3658503643 shutdown_connections 2024-05-17T04:24:58.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 --2- 172.21.15.110:0/3658503643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f2214102a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:58.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 -- 172.21.15.110:0/3658503643 >> 172.21.15.110:0/3658503643 conn(0x7f22140fe0d0 msgr2=0x7f2214100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:58.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 -- 172.21.15.110:0/3658503643 shutdown_connections 2024-05-17T04:24:58.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.084+0000 7f221a06c700 1 -- 172.21.15.110:0/3658503643 wait complete. 2024-05-17T04:24:58.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.085+0000 7f221a06c700 1 Processor -- start 2024-05-17T04:24:58.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.085+0000 7f221a06c700 1 -- start start 2024-05-17T04:24:58.087 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f221a06c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f221a06c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f221410c7f0 con 0x7f2214102650 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f22137fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f22137fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.110:56454/0 (socket says 172.21.15.110:56454) 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f22137fe700 1 -- 172.21.15.110:0/1356094191 learned_addr learned my addr 172.21.15.110:0/1356094191 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f22137fe700 1 -- 172.21.15.110:0/1356094191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221406c730 con 0x7f2214102650 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.086+0000 7f22137fe700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2204007600 tx=0x7f220400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:58.088 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.087+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2204017040 con 0x7f2214102650 2024-05-17T04:24:58.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.087+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2204022070 con 0x7f2214102650 2024-05-17T04:24:58.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.087+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221419a630 con 0x7f2214102650 2024-05-17T04:24:58.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.087+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2204004870 con 0x7f2214102650 2024-05-17T04:24:58.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.087+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221419aaf0 con 0x7f2214102650 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.088+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2214103b00 con 0x7f2214102650 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.088+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2204027460 con 0x7f2214102650 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.088+0000 7f2210ff9700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 0x7f2200040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.088+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f220400f050 con 0x7f2214102650 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.089+0000 7f2212ffd700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 0x7f2200040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:24:58.090 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.089+0000 7f2212ffd700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 0x7f2200040a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:24:58.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.094+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f220401cd10 con 0x7f2214102650 2024-05-17T04:24:58.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.307+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}) -- 0x7f2214102ae0 con 0x7f220003e560 2024-05-17T04:24:58.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.314+0000 7f2210ff9700 1 -- 172.21.15.110:0/1356094191 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2214102ae0 con 0x7f220003e560 2024-05-17T04:24:58.315 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mon update... 2024-05-17T04:24:58.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 msgr2=0x7f2200040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 0x7f2200040a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006fd0 tx=0x7f21fc006e40 comp rx=0 tx=0).stop 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 msgr2=0x7f221419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2204007600 tx=0x7f220400bd40 comp rx=0 tx=0).stop 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 shutdown_connections 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f220003e560 0x7f2200040a70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 --2- 172.21.15.110:0/1356094191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2214102650 0x7f221419a030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 >> 172.21.15.110:0/1356094191 conn(0x7f22140fe0d0 msgr2=0x7f22141087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 shutdown_connections 2024-05-17T04:24:58.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:24:58.317+0000 7f221a06c700 1 -- 172.21.15.110:0/1356094191 wait complete. 2024-05-17T04:24:58.921 DEBUG:teuthology.orchestra.run.smithi105:mon.b> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.b.service 2024-05-17T04:24:58.923 DEBUG:teuthology.orchestra.run.smithi110:mon.c> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.c.service 2024-05-17T04:24:58.926 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T04:24:58.926 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c;count:3 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: Deploying daemon mon.c on smithi110 2024-05-17T04:24:59.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:24:59 smithi050 ceph-mon[31739]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:00.503 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:02.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:01 smithi050 ceph-mon[31739]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:02.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:02 smithi110 systemd[1]: Starting Ceph mon.c for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:25:02.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.672+0000 7f256f5b4700 1 -- 172.21.15.110:0/886568017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 msgr2=0x7f2568102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.672+0000 7f256f5b4700 1 --2- 172.21.15.110:0/886568017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f2568102a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f256400b0a0 tx=0x7f256400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.672+0000 7f256f5b4700 1 -- 172.21.15.110:0/886568017 shutdown_connections 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.672+0000 7f256f5b4700 1 --2- 172.21.15.110:0/886568017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f2568102a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.672+0000 7f256f5b4700 1 -- 172.21.15.110:0/886568017 >> 172.21.15.110:0/886568017 conn(0x7f25680fe100 msgr2=0x7f2568100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.673+0000 7f256f5b4700 1 -- 172.21.15.110:0/886568017 shutdown_connections 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.673+0000 7f256f5b4700 1 -- 172.21.15.110:0/886568017 wait complete. 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.673+0000 7f256f5b4700 1 Processor -- start 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.673+0000 7f256f5b4700 1 -- start start 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256f5b4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:02.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256f5b4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f256810c7c0 con 0x7f2568102620 2024-05-17T04:25:02.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256d350700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:02.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256d350700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.110:56472/0 (socket says 172.21.15.110:56472) 2024-05-17T04:25:02.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256d350700 1 -- 172.21.15.110:0/2909990853 learned_addr learned my addr 172.21.15.110:0/2909990853 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:25:02.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256d350700 1 -- 172.21.15.110:0/2909990853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256806c730 con 0x7f2568102620 2024-05-17T04:25:02.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.674+0000 7f256d350700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2564007600 tx=0x7f256400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:02.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.675+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564017040 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.675+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256819a680 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.675+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2564022070 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.675+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2564004870 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.675+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256819ab40 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.676+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2568103ad0 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.676+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2564027460 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.676+0000 7f255e7fc700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 0x7f2554040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.676+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f256400f050 con 0x7f2568102620 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.676+0000 7f256cb4f700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 0x7f2554040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:02.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.677+0000 7f256cb4f700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 0x7f2554040a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2558006fd0 tx=0x7f2558006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:02.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.681+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f256401ce30 con 0x7f2568102620 2024-05-17T04:25:02.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:02 smithi110 podman[35478]: 2024-05-17 04:25:02.792397705 +0000 UTC m=+0.101336070 container create 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:25:02.824 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:02 smithi110 podman[35478]: 2024-05-17 04:25:02.708183131 +0000 UTC m=+0.017121508 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:25:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.991+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2568102ab0 con 0x7f2568102620 2024-05-17T04:25:02.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.992+0000 7f255e7fc700 1 -- 172.21.15.110:0/2909990853 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2564010db0 con 0x7f2568102620 2024-05-17T04:25:02.993 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:25:02.993 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":1,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","modified":"2024-05-17T04:23:17.907940Z","created":"2024-05-17T04:23:17.907940Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 msgr2=0x7f2554040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 0x7f2554040a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2558006fd0 tx=0x7f2558006e40 comp rx=0 tx=0).stop 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 msgr2=0x7f256819a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2564007600 tx=0x7f256400bd40 comp rx=0 tx=0).stop 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 shutdown_connections 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f255403e560 0x7f2554040a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 --2- 172.21.15.110:0/2909990853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2568102620 0x7f256819a080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 >> 172.21.15.110:0/2909990853 conn(0x7f25680fe100 msgr2=0x7f25681087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 shutdown_connections 2024-05-17T04:25:02.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:02.995+0000 7f256f5b4700 1 -- 172.21.15.110:0/2909990853 wait complete. 2024-05-17T04:25:02.997 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 1 2024-05-17T04:25:03.315 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:02 smithi110 podman[35478]: 2024-05-17 04:25:02.978362481 +0000 UTC m=+0.287300855 container init 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:25:03.315 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:02 smithi110 podman[35478]: 2024-05-17 04:25:02.982194713 +0000 UTC m=+0.291133086 container start 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-17T04:25:03.315 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: set uid:gid to 167:167 (ceph:ceph) 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev), process ceph-mon, pid 2 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pidfile_write: ignore empty --pid-file 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: load: jerasure load: lrc 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: RocksDB version: 7.9.2 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Git sha 0 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Compile date 2024-05-17 01:53:57 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: DB SUMMARY 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: DB Session ID: MXGMP2LL6T62MFHRGGR7 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: CURRENT file: CURRENT 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: IDENTITY file: IDENTITY 2024-05-17T04:25:03.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.error_if_exists: 0 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.create_if_missing: 0 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.paranoid_checks: 1 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.env: 0x56149ab40500 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.fs: PosixFileSystem 2024-05-17T04:25:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.info_log: 0x56149bd30080 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_file_opening_threads: 16 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.statistics: (nil) 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.use_fsync: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_log_file_size: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.keep_log_file_num: 1000 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.recycle_log_file_num: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_fallocate: 1 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_mmap_reads: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_mmap_writes: 0 2024-05-17T04:25:03.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.use_direct_reads: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.create_missing_column_families: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.db_log_dir: 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.wal_dir: 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.advise_random_on_open: 1 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.db_write_buffer_size: 0 2024-05-17T04:25:03.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.write_buffer_manager: 0x56149bbe4b40 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.rate_limiter: (nil) 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.wal_recovery_mode: 2 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enable_thread_tracking: 0 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enable_pipelined_write: 0 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.unordered_write: 0 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-17T04:25:03.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.row_cache: None 2024-05-17T04:25:03.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.wal_filter: None 2024-05-17T04:25:03.322 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-17T04:25:03.322 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_ingest_behind: 0 2024-05-17T04:25:03.322 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.two_write_queues: 0 2024-05-17T04:25:03.322 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.manual_wal_flush: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.wal_compression: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.atomic_flush: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.log_readahead_size: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.best_efforts_recovery: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.allow_data_in_errors: 0 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.db_host_id: __hostname__ 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_background_jobs: 2 2024-05-17T04:25:03.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_background_compactions: -1 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_subcompactions: 1 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_total_wal_size: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_open_files: -1 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bytes_per_sync: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_readahead_size: 0 2024-05-17T04:25:03.324 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_background_flushes: -1 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Compression algorithms supported: 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kZSTD supported: 0 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kXpressCompression supported: 0 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kLZ4HCCompression supported: 1 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kZlibCompression supported: 1 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kSnappyCompression supported: 1 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kLZ4Compression supported: 1 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: kBZip2Compression supported: 0 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-17T04:25:03.325 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.merge_operator: 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_filter: None 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_filter_factory: None 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.sst_partitioner_factory: None 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56149bd30240) 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_type: 0 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_index_type: 0 2024-05-17T04:25:03.326 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_shortening: 1 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: checksum: 4 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: no_block_cache: 0 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache: 0x56149bc2d1f0 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_options: 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: capacity : 536870912 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_shard_bits : 4 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: strict_capacity_limit : 0 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_compressed: (nil) 2024-05-17T04:25:03.327 INFO:journalctl@ceph.mon.c.smithi110.stdout: persistent_cache: (nil) 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size: 4096 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size_deviation: 10 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_restart_interval: 16 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_block_restart_interval: 1 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: metadata_block_size: 4096 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: partition_filters: 0 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: use_delta_encoding: 1 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: filter_policy: bloomfilter 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: whole_key_filtering: 1 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: verify_compression: 0 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-05-17T04:25:03.328 INFO:journalctl@ceph.mon.c.smithi110.stdout: format_version: 5 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_index_compression: 1 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_align: 0 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: max_auto_readahead_size: 262144 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: prepopulate_block_cache: 0 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: initial_auto_readahead_size: 8192 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.write_buffer_size: 33554432 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_write_buffer_number: 2 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression: NoCompression 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression: Disabled 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.prefix_extractor: nullptr 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.num_levels: 7 2024-05-17T04:25:03.329 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-17T04:25:03.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.level: 32767 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.strategy: 0 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.enabled: false 2024-05-17T04:25:03.331 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.target_file_size_base: 67108864 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-17T04:25:03.332 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.arena_block_size: 1048576 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.disable_auto_compactions: 0 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-17T04:25:03.333 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.inplace_update_support: 0 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.bloom_locality: 0 2024-05-17T04:25:03.334 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.max_successive_merges: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.paranoid_file_checks: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.force_consistency_checks: 1 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.report_bg_io_stats: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.ttl: 2592000 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enable_blob_files: false 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.min_blob_size: 0 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_file_size: 268435456 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-17T04:25:03.335 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.blob_file_starting_level: 0 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e1db2080-df3d-4d29-9791-c321c3ca2bf6 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919903025363, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919903025979, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715919903, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e1db2080-df3d-4d29-9791-c321c3ca2bf6", "db_session_id": "MXGMP2LL6T62MFHRGGR7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715919903026103, "job": 1, "event": "recovery_finished"} 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56149bc0fc00 2024-05-17T04:25:03.336 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: DB pointer 0x5614a0a04000 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** DB Stats ** 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-05-17T04:25:03.337 INFO:journalctl@ceph.mon.c.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.338 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval compaction: 0.00 GB write, 0.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache BinnedLRUCache@0x56149bc2d1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-17T04:25:03.339 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: using public_addr v2:172.21.15.110:0/0 -> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: starting mon.c rank -1 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(???) e0 preinit fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 bash[35478]: 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).mds e1 new map 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: e1 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: legacy client fscid: -1 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout: No filesystems configured 2024-05-17T04:25:03.340 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.341 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Deploying cephadm binary to smithi110 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi105:/etc/ceph/ceph.conf 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Added host smithi110 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.343 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2608592261' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 systemd[1]: Started Ceph mon.c for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:03.344 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2608592261' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Saving service mon spec with placement smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c;count:3 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.345 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: Deploying daemon mon.c on smithi110 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:03.346 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:03 smithi110 ceph-mon[35497]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-17T04:25:04.643 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T04:25:04.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T04:25:04.888 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:25:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: mon.a calling monitor election 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: mon.c calling monitor election 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: monmap epoch 2 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: last_changed 2024-05-17T04:25:03.054301+0000 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: min_mon_release 19 (squid) 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: election_strategy: 1 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: fsmap 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: mgrmap e13: x(active, since 49s) 2024-05-17T04:25:08.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:25:08.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:08.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: mon.a calling monitor election 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: mon.c calling monitor election 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: monmap epoch 2 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: last_changed 2024-05-17T04:25:03.054301+0000 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: min_mon_release 19 (squid) 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: election_strategy: 1 2024-05-17T04:25:08.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: fsmap 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: mgrmap e13: x(active, since 49s) 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:09.438 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:09 smithi110 ceph-mon[35497]: Deploying daemon mon.b on smithi105 2024-05-17T04:25:09.438 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:09 smithi110 ceph-mon[35497]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:09.439 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:09.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 -- 172.21.15.110:0/1787213262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 msgr2=0x7fc5dc005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:09.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/1787213262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5dc005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc5f006c310 tx=0x7fc5ec00feb0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 -- 172.21.15.110:0/1787213262 shutdown_connections 2024-05-17T04:25:09.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/1787213262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5dc005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 -- 172.21.15.110:0/1787213262 >> 172.21.15.110:0/1787213262 conn(0x7fc5f00fe1d0 msgr2=0x7fc5f0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:09.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 -- 172.21.15.110:0/1787213262 shutdown_connections 2024-05-17T04:25:09.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.437+0000 7fc5f81ca700 1 -- 172.21.15.110:0/1787213262 wait complete. 2024-05-17T04:25:09.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.438+0000 7fc5f81ca700 1 Processor -- start 2024-05-17T04:25:09.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.438+0000 7fc5f81ca700 1 -- start start 2024-05-17T04:25:09.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f81ca700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5f01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:09.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f81ca700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:09.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f81ca700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010b440 con 0x7fc5dc0035c0 2024-05-17T04:25:09.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f81ca700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010b2c0 con 0x7fc5f01998e0 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43902/0 (socket says 172.21.15.110:43902) 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 -- 172.21.15.110:0/233416687 learned_addr learned my addr 172.21.15.110:0/233416687 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 -- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 msgr2=0x7fc5f019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 -- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 msgr2=0x7fc5f019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5765700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.439+0000 7fc5f5f66700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5f01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f5f66700 1 -- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 msgr2=0x7fc5f019dbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f5f66700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f5f66700 1 -- 172.21.15.110:0/233416687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f0199ee0 con 0x7fc5dc0035c0 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f5f66700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5f01993a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec000c00 tx=0x7fc5ec005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:09.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc5ec013960 con 0x7fc5dc0035c0 2024-05-17T04:25:09.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ec004030 con 0x7fc5dc0035c0 2024-05-17T04:25:09.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc5ec003b80 con 0x7fc5dc0035c0 2024-05-17T04:25:09.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f019e110 con 0x7fc5dc0035c0 2024-05-17T04:25:09.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.440+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f019e6a0 con 0x7fc5dc0035c0 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.441+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc5ec023070 con 0x7fc5dc0035c0 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.441+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f006c5c0 con 0x7fc5dc0035c0 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.442+0000 7fc5e6ffd700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 0x7fc5d4040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.442+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc5ec020050 con 0x7fc5dc0035c0 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.442+0000 7fc5f5765700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 0x7fc5d4040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:09.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.443+0000 7fc5f5765700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 0x7fc5d4040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0109400 tx=0x7fc5e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:09.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.447+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5ec028020 con 0x7fc5dc0035c0 2024-05-17T04:25:09.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:09 smithi050 ceph-mon[31739]: Deploying daemon mon.b on smithi105 2024-05-17T04:25:09.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:09 smithi050 ceph-mon[31739]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:09.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:09.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.741+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc5f019a700 con 0x7fc5dc0035c0 2024-05-17T04:25:09.742 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.742+0000 7fc5e6ffd700 1 -- 172.21.15.110:0/233416687 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc5ec0103d0 con 0x7fc5dc0035c0 2024-05-17T04:25:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:25:09.743 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","modified":"2024-05-17T04:25:03.054301Z","created":"2024-05-17T04:23:17.907940Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 msgr2=0x7fc5d4040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 0x7fc5d4040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0109400 tx=0x7fc5e0008040 comp rx=0 tx=0).stop 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 msgr2=0x7fc5f01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5f01993a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec000c00 tx=0x7fc5ec005c70 comp rx=0 tx=0).stop 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 shutdown_connections 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc5d403e300 0x7fc5d4040810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5dc0035c0 0x7fc5f01993a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 --2- 172.21.15.110:0/233416687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5f01998e0 0x7fc5f019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 >> 172.21.15.110:0/233416687 conn(0x7fc5f00fe1d0 msgr2=0x7fc5f01012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 shutdown_connections 2024-05-17T04:25:09.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:09.744+0000 7fc5f81ca700 1 -- 172.21.15.110:0/233416687 wait complete. 2024-05-17T04:25:09.746 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 2 2024-05-17T04:25:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:10 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/233416687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T04:25:10.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:10 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/233416687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T04:25:11.241 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-17T04:25:11.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-17T04:25:11.486 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:25:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:11 smithi110 ceph-mon[35497]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:11.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:11 smithi050 ceph-mon[31739]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:12.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:11 smithi105 systemd[1]: Started Ceph mon.b for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: mon.a calling monitor election 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: mon.c calling monitor election 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:17.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: monmap epoch 3 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: min_mon_release 19 (squid) 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: election_strategy: 1 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: fsmap 2024-05-17T04:25:17.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:17.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: mgrmap e13: x(active, since 58s) 2024-05-17T04:25:17.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:25:17.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:17.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: mon.a calling monitor election 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: mon.c calling monitor election 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:17.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: monmap epoch 3 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: min_mon_release 19 (squid) 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: election_strategy: 1 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: fsmap 2024-05-17T04:25:17.328 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:17.329 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: mgrmap e13: x(active, since 58s) 2024-05-17T04:25:17.329 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:25:17.329 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:17.329 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:17.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/1577130692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf600035c0 msgr2=0x7fbf60005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/1577130692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf600035c0 0x7fbf60005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbf781030c0 tx=0x7fbf68010e70 comp rx=0 tx=0).stop 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/1577130692 shutdown_connections 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/1577130692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf600035c0 0x7fbf60005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/1577130692 >> 172.21.15.110:0/1577130692 conn(0x7fbf780779d0 msgr2=0x7fbf78077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/1577130692 shutdown_connections 2024-05-17T04:25:17.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.331+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/1577130692 wait complete. 2024-05-17T04:25:17.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 Processor -- start 2024-05-17T04:25:17.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 -- start start 2024-05-17T04:25:17.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 0x7fbf7819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf781092f0 con 0x7fbf7819e390 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78109170 con 0x7fbf78199b60 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf7e6cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78109470 con 0x7fbf600035c0 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50090/0 (socket says 172.21.15.110:50090) 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 -- 172.21.15.110:0/822697683 learned_addr learned my addr 172.21.15.110:0/822697683 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 msgr2=0x7fbf78199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-17T04:25:17.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 msgr2=0x7fbf78199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.332+0000 7fbf77fff700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf77fff700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf777fe700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 0x7fbf7819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf777fe700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 msgr2=0x7fbf7819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf777fe700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 msgr2=0x7fbf7819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf777fe700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf777fe700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 msgr2=0x7fbf78199620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 msgr2=0x7fbf7819de50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-17T04:25:17.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 -- 172.21.15.110:0/822697683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf781a6100 con 0x7fbf7819e390 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7cc68700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 0x7fbf7819e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000bd50 tx=0x7fbf7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70019070 con 0x7fbf7819e390 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf7000cd10 con 0x7fbf7819e390 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700125e0 con 0x7fbf7819e390 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.333+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf781a6330 con 0x7fbf7819e390 2024-05-17T04:25:17.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.334+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781a68a0 con 0x7fbf7819e390 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.334+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7810a380 con 0x7fbf7819e390 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.335+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbf7001c070 con 0x7fbf7819e390 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.335+0000 7fbf757fa700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 0x7fbf64040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.335+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbf70052060 con 0x7fbf7819e390 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.335+0000 7fbf77fff700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 0x7fbf64040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:17.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.335+0000 7fbf77fff700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 0x7fbf64040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf68010550 tx=0x7fbf68001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:17.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.340+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf70022170 con 0x7fbf7819e390 2024-05-17T04:25:17.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.640+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbf7819a980 con 0x7fbf7819e390 2024-05-17T04:25:17.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.641+0000 7fbf757fa700 1 -- 172.21.15.110:0/822697683 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fbf700149f0 con 0x7fbf7819e390 2024-05-17T04:25:17.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-17T04:25:17.642 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":3,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","modified":"2024-05-17T04:25:11.934382Z","created":"2024-05-17T04:23:17.907940Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 msgr2=0x7fbf64040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 0x7fbf64040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf68010550 tx=0x7fbf68001e70 comp rx=0 tx=0).stop 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 msgr2=0x7fbf7819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 0x7fbf7819e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000bd50 tx=0x7fbf7000be30 comp rx=0 tx=0).stop 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 shutdown_connections 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbf6403e9a0 0x7fbf64040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf7819e390 0x7fbf7819e7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf600035c0 0x7fbf78199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 --2- 172.21.15.110:0/822697683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf78199b60 0x7fbf7819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:17.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 >> 172.21.15.110:0/822697683 conn(0x7fbf780779d0 msgr2=0x7fbf78101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:17.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 shutdown_connections 2024-05-17T04:25:17.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:17.644+0000 7fbf7e6cb700 1 -- 172.21.15.110:0/822697683 wait complete. 2024-05-17T04:25:17.646 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 3 2024-05-17T04:25:18.161 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-17T04:25:18.162 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-17T04:25:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:17 smithi110 ceph-mon[35497]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:18.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:18.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:18.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:18 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/822697683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T04:25:18.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:18.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:18.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/822697683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T04:25:18.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:18.424 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:25:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: mon.a calling monitor election 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: mon.c calling monitor election 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: monmap epoch 3 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: min_mon_release 19 (squid) 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: election_strategy: 1 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: fsmap 2024-05-17T04:25:19.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: mgrmap e13: x(active, since 58s) 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: overall HEALTH_OK 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/822697683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-17T04:25:19.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:19.958 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mon.b calling monitor election 2024-05-17T04:25:19.958 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: Updating smithi105:/etc/ceph/ceph.conf 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mon.b calling monitor election 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mon.c calling monitor election 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mon.a calling monitor election 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: monmap epoch 3 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: min_mon_release 19 (squid) 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: election_strategy: 1 2024-05-17T04:25:19.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: fsmap 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: mgrmap e13: x(active, since 60s) 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:19.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:20.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 -- 172.21.15.50:0/3884395316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01046b0 msgr2=0x7f80c0104ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/3884395316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01046b0 0x7f80c0104ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009910 tx=0x7f80b0009c20 comp rx=0 tx=0).stop 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 -- 172.21.15.50:0/3884395316 shutdown_connections 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/3884395316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01046b0 0x7f80c0104ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 -- 172.21.15.50:0/3884395316 >> 172.21.15.50:0/3884395316 conn(0x7f80c00fe0b0 msgr2=0x7f80c0100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 -- 172.21.15.50:0/3884395316 shutdown_connections 2024-05-17T04:25:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.180+0000 7f80c4ac1700 1 -- 172.21.15.50:0/3884395316 wait complete. 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.181+0000 7f80c4ac1700 1 Processor -- start 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.181+0000 7f80c4ac1700 1 -- start start 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 0x7f80c01a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 0x7f80c019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010d910 con 0x7f80c01a19b0 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010d790 con 0x7f80c019bb90 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80c4ac1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010da90 con 0x7f80c01046b0 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57468/0 (socket says 172.21.15.50:57468) 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 -- 172.21.15.50:0/1600790097 learned_addr learned my addr 172.21.15.50:0/1600790097 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:25:20.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80be59c700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 0x7f80c01a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bed9d700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 0x7f80c019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 -- 172.21.15.50:0/1600790097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 msgr2=0x7f80c01a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 0x7f80c01a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 -- 172.21.15.50:0/1600790097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 msgr2=0x7f80c019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 0x7f80c019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bdd9b700 1 -- 172.21.15.50:0/1600790097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c01a8270 con 0x7f80c01a19b0 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80bed9d700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 0x7f80c019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.182+0000 7f80be59c700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 0x7f80c01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80bdd9b700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80b400b780 tx=0x7f80b400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4004230 con 0x7f80c01a19b0 2024-05-17T04:25:20.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80b40043d0 con 0x7f80c01a19b0 2024-05-17T04:25:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c01a8560 con 0x7f80c01a19b0 2024-05-17T04:25:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4011a60 con 0x7f80c01a19b0 2024-05-17T04:25:20.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.183+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a8aa0 con 0x7f80c01a19b0 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.184+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c006c100 con 0x7f80c01a19b0 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.184+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f80b4011c00 con 0x7f80c01a19b0 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.185+0000 7f80af7fe700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 0x7f80a8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.185+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f80b40522b0 con 0x7f80c01a19b0 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.185+0000 7f80be59c700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 0x7f80a8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:20.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.185+0000 7f80be59c700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 0x7f80a8040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80b0004690 tx=0x7f80b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:20.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.192+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80b4017d50 con 0x7f80c01a19b0 2024-05-17T04:25:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mon.b calling monitor election 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: Updating smithi105:/etc/ceph/ceph.conf 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mon.b calling monitor election 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mon.c calling monitor election 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mon.a calling monitor election 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: monmap epoch 3 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: min_mon_release 19 (squid) 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: election_strategy: 1 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: fsmap 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: mgrmap e13: x(active, since 60s) 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: overall HEALTH_OK 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.317 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:20.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mon.b calling monitor election 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: Updating smithi105:/etc/ceph/ceph.conf 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mon.b calling monitor election 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mon.c calling monitor election 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mon.a calling monitor election 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: monmap epoch 3 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: last_changed 2024-05-17T04:25:11.934382+0000 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: created 2024-05-17T04:23:17.907940+0000 2024-05-17T04:25:20.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: min_mon_release 19 (squid) 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: election_strategy: 1 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: 0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: fsmap 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: osdmap e4: 0 total, 0 up, 0 in 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: mgrmap e13: x(active, since 60s) 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:20.321 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:20.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.409+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f80c019cc00 con 0x7f80c01a19b0 2024-05-17T04:25:20.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.410+0000 7f80af7fe700 1 -- 172.21.15.50:0/1600790097 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f80b40193a0 con 0x7f80c01a19b0 2024-05-17T04:25:20.411 INFO:teuthology.orchestra.run.smithi050.stdout:# minimal ceph.conf for 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:20.411 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-05-17T04:25:20.411 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:25:20.411 INFO:teuthology.orchestra.run.smithi050.stdout: mon_host = [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-05-17T04:25:20.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 msgr2=0x7f80a8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:20.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 0x7f80a8040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80b0004690 tx=0x7f80b0005fb0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 msgr2=0x7f80c019b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:20.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80b400b780 tx=0x7f80b400ba90 comp rx=0 tx=0).stop 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 shutdown_connections 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80c019bb90 0x7f80c019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c01046b0 0x7f80c01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80a803e9a0 0x7f80a8040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 --2- 172.21.15.50:0/1600790097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80c01a19b0 0x7f80c019b650 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 >> 172.21.15.50:0/1600790097 conn(0x7f80c00fe0b0 msgr2=0x7f80c0109960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 shutdown_connections 2024-05-17T04:25:20.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:20.412+0000 7f80c4ac1700 1 -- 172.21.15.50:0/1600790097 wait complete. 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: Updating smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: Reconfiguring daemon mon.a on smithi050 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:21.151 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:20 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1600790097' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:21.185 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-17T04:25:21.185 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:25:21.185 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T04:25:21.217 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:25:21.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:25:21.285 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:25:21.286 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T04:25:21.290 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.290 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: Updating smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.290 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.291 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T04:25:21.291 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: Reconfiguring daemon mon.a on smithi050 2024-05-17T04:25:21.291 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:21.291 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:20 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1600790097' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:21.316 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:25:21.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:25:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: Updating smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: Updating smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: Updating smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-17T04:25:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: Reconfiguring mon.a (unknown last config time)... 2024-05-17T04:25:21.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: Reconfiguring daemon mon.a on smithi050 2024-05-17T04:25:21.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:25:21.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:20 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1600790097' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:21.383 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:25:21.383 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-05-17T04:25:21.416 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:25:21.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-17T04:25:21.487 INFO:tasks.cephadm:Adding mgr.x on smithi050 2024-05-17T04:25:21.487 INFO:tasks.cephadm:Adding mgr.y on smithi105 2024-05-17T04:25:21.487 INFO:tasks.cephadm:Adding mgr.z on smithi110 2024-05-17T04:25:21.487 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi050=x;smithi105=y;smithi110=z' 2024-05-17T04:25:21.764 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:25:22.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:21 smithi050 ceph-mon[31739]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:22.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:21 smithi110 ceph-mon[35497]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:21 smithi105 ceph-mon[34795]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:23.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: Reconfiguring mon.b (monmap changed)... 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:23.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:23 smithi110 ceph-mon[35497]: Reconfiguring daemon mon.b on smithi105 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 -- 172.21.15.110:0/1152400018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec102b30 msgr2=0x7f10ec102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/1152400018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec102b30 0x7f10ec102f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009910 tx=0x7f10dc009c20 comp rx=0 tx=0).stop 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 -- 172.21.15.110:0/1152400018 shutdown_connections 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/1152400018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec102b30 0x7f10ec102f40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 -- 172.21.15.110:0/1152400018 >> 172.21.15.110:0/1152400018 conn(0x7f10ec0fd530 msgr2=0x7f10ec0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 -- 172.21.15.110:0/1152400018 shutdown_connections 2024-05-17T04:25:23.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.305+0000 7f10f1f8a700 1 -- 172.21.15.110:0/1152400018 wait complete. 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 Processor -- start 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 -- start start 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10ec102b30 0x7f10ec1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 0x7f10ec1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec108390 con 0x7f10ec19e920 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec108210 con 0x7f10ec102b30 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10f1f8a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec108510 con 0x7f10ec19bfc0 2024-05-17T04:25:23.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50104/0 (socket says 172.21.15.110:50104) 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 -- 172.21.15.110:0/162313051 learned_addr learned my addr 172.21.15.110:0/162313051 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10eb7fe700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10ec102b30 0x7f10ec1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10ebfff700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 0x7f10ec1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 -- 172.21.15.110:0/162313051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10ec102b30 msgr2=0x7f10ec1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10ec102b30 0x7f10ec1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 -- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 msgr2=0x7f10ec1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 0x7f10ec1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10e3fff700 1 -- 172.21.15.110:0/162313051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec1a1590 con 0x7f10ec19bfc0 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.306+0000 7f10ebfff700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 0x7f10ec1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10e3fff700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b780 tx=0x7f10d400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:23.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004230 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec1a1880 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10d40043d0 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4011a10 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.307+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1a1e90 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.308+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f10d4011bf0 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.308+0000 7f10e97fa700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 0x7f10d8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.308+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec102fc0 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.308+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f10d4003b90 con 0x7f10ec19bfc0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.308+0000 7f10eb7fe700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 0x7f10d8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:23.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.309+0000 7f10eb7fe700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 0x7f10d8040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009750 tx=0x7f10dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:23.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.313+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10d4020740 con 0x7f10ec19bfc0 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: Reconfiguring mon.b (monmap changed)... 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:23 smithi105 ceph-mon[34795]: Reconfiguring daemon mon.b on smithi105 2024-05-17T04:25:23.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:23.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: Reconfiguring mon.b (monmap changed)... 2024-05-17T04:25:23.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:23.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:23.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:23.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:23 smithi050 ceph-mon[31739]: Reconfiguring daemon mon.b on smithi105 2024-05-17T04:25:23.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.526+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi105=y;smithi110=z", "target": ["mon-mgr", ""]}) -- 0x7f10ec0ff4c0 con 0x7f10d803e9a0 2024-05-17T04:25:23.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.533+0000 7f10e97fa700 1 -- 172.21.15.110:0/162313051 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f10ec0ff4c0 con 0x7f10d803e9a0 2024-05-17T04:25:23.534 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mgr update... 2024-05-17T04:25:23.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 msgr2=0x7f10d8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 0x7f10d8040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009750 tx=0x7f10dc01a040 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 msgr2=0x7f10ec19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b780 tx=0x7f10d400ba90 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 shutdown_connections 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10d803e9a0 0x7f10d8040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10ec19e920 0x7f10ec1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10ec102b30 0x7f10ec1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 --2- 172.21.15.110:0/162313051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10ec19bfc0 0x7f10ec19e3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 >> 172.21.15.110:0/162313051 conn(0x7f10ec0fd530 msgr2=0x7f10ec0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 shutdown_connections 2024-05-17T04:25:23.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:25:23.536+0000 7f10f1f8a700 1 -- 172.21.15.110:0/162313051 wait complete. 2024-05-17T04:25:24.140 DEBUG:teuthology.orchestra.run.smithi105:mgr.y> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-17T04:25:24.143 DEBUG:teuthology.orchestra.run.smithi110:mgr.z> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-17T04:25:24.145 INFO:tasks.cephadm:Deploying OSDs... 2024-05-17T04:25:24.145 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:25:24.145 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T04:25:24.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T04:25:24.166 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-05-17T04:25:24.167 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:24.167 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.167 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.167 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:24.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:24.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:24.225 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T04:25:24.225 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-17 04:24:28.865278458 +0000 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-17 04:19:44.128480936 +0000 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-17 04:19:44.128480936 +0000 2024-05-17T04:25:24.226 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-17 04:19:44.128480936 +0000 2024-05-17T04:25:24.226 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T04:25:24.294 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-17T04:25:24.295 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-17T04:25:24.296 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000113798 s, 4.5 MB/s 2024-05-17T04:25:24.297 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:24.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:24.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-17T04:25:24.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-17T04:25:24.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:24.345 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-17 04:24:28.901277575 +0000 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-17 04:19:44.482472139 +0000 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-17 04:19:44.482472139 +0000 2024-05-17T04:25:24.403 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-17 04:19:44.482472139 +0000 2024-05-17T04:25:24.404 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T04:25:24.469 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-17T04:25:24.470 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-17T04:25:24.470 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000118362 s, 4.3 MB/s 2024-05-17T04:25:24.471 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T04:25:24.529 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-05-17T04:25:24.587 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-17 04:24:28.935276741 +0000 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-17 04:19:44.807464063 +0000 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-17 04:19:44.807464063 +0000 2024-05-17T04:25:24.588 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-17 04:19:44.807464063 +0000 2024-05-17T04:25:24.588 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T04:25:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-17T04:25:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-17T04:25:24.654 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000107648 s, 4.8 MB/s 2024-05-17T04:25:24.655 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T04:25:24.713 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:24.770 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-17 04:24:28.969275907 +0000 2024-05-17T04:25:24.771 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-17 04:19:45.135455912 +0000 2024-05-17T04:25:24.771 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-17 04:19:45.135455912 +0000 2024-05-17T04:25:24.771 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-17 04:19:45.135455912 +0000 2024-05-17T04:25:24.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T04:25:24.835 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-17T04:25:24.836 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-17T04:25:24.836 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000114026 s, 4.5 MB/s 2024-05-17T04:25:24.837 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T04:25:24.895 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-17T04:25:24.895 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T04:25:24.914 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T04:25:24.914 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-05-17T04:25:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T04:25:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:24.973 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-17 04:25:11.358744247 +0000 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-17 04:19:39.010410520 +0000 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-17 04:19:39.010410520 +0000 2024-05-17T04:25:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-17 04:19:39.010410520 +0000 2024-05-17T04:25:24.974 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T04:25:25.041 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-17T04:25:25.041 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-17T04:25:25.041 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000227006 s, 2.3 MB/s 2024-05-17T04:25:25.042 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T04:25:25.101 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-17 04:25:11.359744229 +0000 2024-05-17T04:25:25.159 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-17 04:19:39.371403498 +0000 2024-05-17T04:25:25.160 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-17 04:19:39.371403498 +0000 2024-05-17T04:25:25.160 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-17 04:19:39.371403498 +0000 2024-05-17T04:25:25.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T04:25:25.225 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-17T04:25:25.225 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-17T04:25:25.225 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000226715 s, 2.3 MB/s 2024-05-17T04:25:25.226 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T04:25:25.284 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-05-17T04:25:25.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:25 smithi105 ceph-mon[34795]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi105=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:25.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:25 smithi105 ceph-mon[34795]: Saving service mgr spec with placement smithi050=x;smithi105=y;smithi110=z;count:3 2024-05-17T04:25:25.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:25 smithi105 ceph-mon[34795]: Reconfiguring mon.c (monmap changed)... 2024-05-17T04:25:25.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:25 smithi105 ceph-mon[34795]: Reconfiguring daemon mon.c on smithi110 2024-05-17T04:25:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:25 smithi110 ceph-mon[35497]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi105=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:25 smithi110 ceph-mon[35497]: Saving service mgr spec with placement smithi050=x;smithi105=y;smithi110=z;count:3 2024-05-17T04:25:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:25 smithi110 ceph-mon[35497]: Reconfiguring mon.c (monmap changed)... 2024-05-17T04:25:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:25 smithi110 ceph-mon[35497]: Reconfiguring daemon mon.c on smithi110 2024-05-17T04:25:25.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:25 smithi050 ceph-mon[31739]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi050=x;smithi105=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:25.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:25 smithi050 ceph-mon[31739]: Saving service mgr spec with placement smithi050=x;smithi105=y;smithi110=z;count:3 2024-05-17T04:25:25.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:25 smithi050 ceph-mon[31739]: Reconfiguring mon.c (monmap changed)... 2024-05-17T04:25:25.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:25 smithi050 ceph-mon[31739]: Reconfiguring daemon mon.c on smithi110 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-17 04:25:11.359744229 +0000 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-17 04:19:39.692397254 +0000 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-17 04:19:39.692397254 +0000 2024-05-17T04:25:25.331 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-17 04:19:39.692397254 +0000 2024-05-17T04:25:25.332 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T04:25:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-17T04:25:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-17T04:25:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000224363 s, 2.3 MB/s 2024-05-17T04:25:25.398 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T04:25:25.455 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-17 04:25:11.359744229 +0000 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-17 04:19:40.023390815 +0000 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-17 04:19:40.023390815 +0000 2024-05-17T04:25:25.512 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-17 04:19:40.023390815 +0000 2024-05-17T04:25:25.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T04:25:25.576 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-17T04:25:25.576 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-17T04:25:25.576 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000175353 s, 2.9 MB/s 2024-05-17T04:25:25.577 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T04:25:25.633 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-17T04:25:25.633 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T04:25:25.653 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T04:25:25.654 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:25.712 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-17 04:25:01.970464075 +0000 2024-05-17T04:25:25.713 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-17 04:19:38.996016262 +0000 2024-05-17T04:25:25.713 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-17 04:19:38.996016262 +0000 2024-05-17T04:25:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-17 04:19:38.996016262 +0000 2024-05-17T04:25:25.713 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T04:25:25.780 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-17T04:25:25.780 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-17T04:25:25.780 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.9239e-05 s, 5.2 MB/s 2024-05-17T04:25:25.782 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T04:25:25.839 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-05-17T04:25:25.895 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T04:25:25.895 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:25.895 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-17T04:25:25.895 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:25.895 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:25.896 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-17 04:25:01.971464059 +0000 2024-05-17T04:25:25.896 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-17 04:19:39.309011098 +0000 2024-05-17T04:25:25.896 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-17 04:19:39.309011098 +0000 2024-05-17T04:25:25.896 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-17 04:19:39.309011098 +0000 2024-05-17T04:25:25.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T04:25:25.959 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-17T04:25:25.959 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-17T04:25:25.959 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000112895 s, 4.5 MB/s 2024-05-17T04:25:25.960 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T04:25:26.016 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-17 04:25:01.971464059 +0000 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-17 04:19:39.620005967 +0000 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-17 04:19:39.620005967 +0000 2024-05-17T04:25:26.072 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-17 04:19:39.620005967 +0000 2024-05-17T04:25:26.073 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T04:25:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-17T04:25:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-17T04:25:26.136 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.7242e-05 s, 5.9 MB/s 2024-05-17T04:25:26.137 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T04:25:26.193 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-17 04:25:01.971464059 +0000 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-17 04:19:39.947000572 +0000 2024-05-17T04:25:26.249 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-17 04:19:39.947000572 +0000 2024-05-17T04:25:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-17 04:19:39.947000572 +0000 2024-05-17T04:25:26.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.311 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:26.312 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.312 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:26.312 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:26.312 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:26.312 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.313 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-17T04:25:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-17T04:25:26.314 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.2579e-05 s, 5.5 MB/s 2024-05-17T04:25:26.315 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:26.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:26.372 INFO:tasks.cephadm:Deploying osd.0 on smithi050 with /dev/vg_nvme/lv_4... 2024-05-17T04:25:26.372 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T04:25:26.625 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:25:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:27 smithi105 ceph-mon[34795]: Deploying daemon mgr.z on smithi110 2024-05-17T04:25:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:27 smithi110 ceph-mon[35497]: Deploying daemon mgr.z on smithi110 2024-05-17T04:25:27.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:27 smithi050 ceph-mon[31739]: Deploying daemon mgr.z on smithi110 2024-05-17T04:25:27.818 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:27 smithi110 systemd[1]: Starting Ceph mgr.z for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:25:28.074 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:27 smithi110 podman[37334]: 2024-05-17 04:25:27.72440696 +0000 UTC m=+0.017632554 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:25:28.074 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:27 smithi110 podman[37334]: 2024-05-17 04:25:27.826921125 +0000 UTC m=+0.120146706 container create 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-17T04:25:28.074 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 podman[37334]: 2024-05-17 04:25:28.037704722 +0000 UTC m=+0.330930302 container init 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-17T04:25:28.074 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 podman[37334]: 2024-05-17 04:25:28.040653666 +0000 UTC m=+0.333879257 container start 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-17T04:25:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:28 smithi050 ceph-mon[31739]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:28.391 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:28 smithi110 ceph-mon[35497]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:28.391 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-mgr[37353]: -- 172.21.15.110:0/2434459564 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a59e9c5a0 con 0x557a5ec26480 2024-05-17T04:25:28.391 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 bash[37334]: 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 2024-05-17T04:25:28.392 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 systemd[1]: Started Ceph mgr.z for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:25:28.392 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:28.206+0000 7f10e2136200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T04:25:28.392 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:28.390+0000 7f10e2136200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:25:28.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:28 smithi105 ceph-mon[34795]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T04:25:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T04:25:28.602 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-17T04:25:28.602 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-17T04:25:28.603 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00845688 s, 1.2 GB/s 2024-05-17T04:25:28.603 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 2024-05-17T04:25:28.603 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-17T04:25:28.795 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:28.491+0000 7f10e2136200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T04:25:28.796 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:28.795+0000 7f10e2136200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T04:25:29.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:28.879+0000 7f10e2136200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T04:25:29.300 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.300 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.300 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.300 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:29.301 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:29.301 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:29.301 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:29.301 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: Deploying daemon mgr.y on smithi105 2024-05-17T04:25:29.301 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:29 smithi050 ceph-mon[31739]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:29.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:29.498 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:29.499 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: Deploying daemon mgr.y on smithi105 2024-05-17T04:25:29.499 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:29 smithi110 ceph-mon[35497]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:29.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:29.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: Deploying daemon mgr.y on smithi105 2024-05-17T04:25:29.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:29 smithi105 ceph-mon[34795]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:29.590 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:25:30.315 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 systemd[1]: Starting Ceph mgr.y for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:25:30.446 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:30.115+0000 7f10e2136200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T04:25:30.446 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:30.284+0000 7f10e2136200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T04:25:30.675 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 podman[36153]: 2024-05-17 04:25:30.313842448 +0000 UTC m=+0.017031587 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:25:30.675 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 podman[36153]: 2024-05-17 04:25:30.414609615 +0000 UTC m=+0.117798782 container create 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-17T04:25:30.676 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 podman[36153]: 2024-05-17 04:25:30.600261846 +0000 UTC m=+0.303450976 container init 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T04:25:30.676 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 podman[36153]: 2024-05-17 04:25:30.603824465 +0000 UTC m=+0.307013592 container start 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-17T04:25:30.676 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 ceph-mgr[36172]: -- 172.21.15.105:0/1452123726 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b18040a5a0 con 0x55b185194000 2024-05-17T04:25:30.676 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 bash[36153]: 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 2024-05-17T04:25:30.746 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:30.455+0000 7f10e2136200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T04:25:30.746 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:30.745+0000 7f10e2136200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T04:25:30.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 systemd[1]: Started Ceph mgr.y for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:25:30.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:30.769+0000 7f7dad75b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T04:25:30.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:30.958+0000 7f7dad75b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:25:31.045 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:30.838+0000 7f10e2136200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T04:25:31.045 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:31.044+0000 7f10e2136200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T04:25:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 -- 172.21.15.50:0/3655626367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec1030f0 msgr2=0x7fb8ec103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/3655626367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec103500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009990 tx=0x7fb8e8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 -- 172.21.15.50:0/3655626367 shutdown_connections 2024-05-17T04:25:31.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/3655626367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec104620 0x7fb8ec10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/3655626367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec103500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/3655626367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 -- 172.21.15.50:0/3655626367 >> 172.21.15.50:0/3655626367 conn(0x7fb8ec0fdb50 msgr2=0x7fb8ec0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 -- 172.21.15.50:0/3655626367 shutdown_connections 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.185+0000 7fb8f42f9700 1 -- 172.21.15.50:0/3655626367 wait complete. 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 Processor -- start 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 -- start start 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec10d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec10ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec10f5e0 con 0x7fb8ec104620 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec10f460 con 0x7fb8ec103ad0 2024-05-17T04:25:31.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f42f9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec10f760 con 0x7fb8ec1030f0 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f2896700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f2095700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec10d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f2896700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41630/0 (socket says 172.21.15.50:41630) 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f2896700 1 -- 172.21.15.50:0/4207721356 learned_addr learned my addr 172.21.15.50:0/4207721356 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.186+0000 7fb8f1894700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec10ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 -- 172.21.15.50:0/4207721356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec1030f0 msgr2=0x7fb8ec10d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec10d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 -- 172.21.15.50:0/4207721356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 msgr2=0x7fb8ec10ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec10ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 -- 172.21.15.50:0/4207721356 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec10c6c0 con 0x7fb8ec104620 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f1894700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:25:31.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f2896700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ef30 tx=0x7fb8e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40092e0 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8e4004bb0 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1a4330 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4004020 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.187+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1a4790 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.188+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec103500 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.188+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb8e401b030 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.188+0000 7fb8e37fe700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 0x7fb8d8040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.189+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8e4052570 con 0x7fb8ec104620 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.189+0000 7fb8f2095700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 0x7fb8d8040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:31.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.189+0000 7fb8f2095700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 0x7fb8d8040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b650 tx=0x7fb8e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:31.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.193+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8e4022020 con 0x7fb8ec104620 2024-05-17T04:25:31.291 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:31.069+0000 7f7dad75b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T04:25:31.317 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:31.247+0000 7f10e2136200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:25:31.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:31.419+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb8ec0646a0 con 0x7fb8d803e750 2024-05-17T04:25:31.565 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:31.400+0000 7f7dad75b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T04:25:31.565 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:31.495+0000 7f7dad75b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:32.053 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:32.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:31.891+0000 7f10e2136200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T04:25:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:32.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:32.549 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:32.117+0000 7f10e2136200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T04:25:32.817 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:32.548+0000 7f10e2136200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T04:25:32.817 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:32.647+0000 7f10e2136200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:32.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:32.819 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:33.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:33.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:33.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:32.781+0000 7f7dad75b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:32.947+0000 7f7dad75b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-17T04:25:33.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:33.317 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.016+0000 7f10e2136200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T04:25:33.397 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:33.115+0000 7f7dad75b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T04:25:33.397 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:33.396+0000 7f7dad75b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T04:25:33.687 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:33.487+0000 7f7dad75b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T04:25:33.687 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:33.686+0000 7f7dad75b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T04:25:33.701 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.367+0000 7f10e2136200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T04:25:33.701 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.459+0000 7f10e2136200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T04:25:34.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:33 smithi105 ceph-mon[34795]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:34.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:33 smithi105 ceph-mon[34795]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T04:25:34.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:33 smithi105 ceph-mon[34795]: Reconfiguring daemon mgr.x on smithi050 2024-05-17T04:25:34.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:34.065 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:33.877+0000 7f7dad75b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T04:25:34.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.700+0000 7f10e2136200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T04:25:34.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.833+0000 7f10e2136200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T04:25:34.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:33.929+0000 7f10e2136200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T04:25:34.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:33 smithi110 ceph-mon[35497]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:34.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:33 smithi110 ceph-mon[35497]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T04:25:34.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:33 smithi110 ceph-mon[35497]: Reconfiguring daemon mgr.x on smithi050 2024-05-17T04:25:34.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:33 smithi050 ceph-mon[31739]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:33 smithi050 ceph-mon[31739]: Reconfiguring mgr.x (unknown last config time)... 2024-05-17T04:25:34.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:33 smithi050 ceph-mon[31739]: Reconfiguring daemon mgr.x on smithi050 2024-05-17T04:25:34.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:34.569 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:34.254+0000 7f10e2136200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T04:25:34.570 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:34.346+0000 7f10e2136200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T04:25:34.748 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:34.468+0000 7f7dad75b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-17T04:25:34.748 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:34.683+0000 7f7dad75b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-17T04:25:35.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:34 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]: dispatch 2024-05-17T04:25:35.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:34 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]': finished 2024-05-17T04:25:35.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:34 smithi105 ceph-mon[34795]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T04:25:35.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:35.067 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:34.943+0000 7f10e2136200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T04:25:35.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:34 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]: dispatch 2024-05-17T04:25:35.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:34 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]': finished 2024-05-17T04:25:35.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:34 smithi110 ceph-mon[35497]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T04:25:35.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:35.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:34 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]: dispatch 2024-05-17T04:25:35.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:34 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1688123836' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126dad7f-7abd-4e39-8390-c63a919ff76f"}]': finished 2024-05-17T04:25:35.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:34 smithi050 ceph-mon[31739]: osdmap e5: 1 total, 0 up, 1 in 2024-05-17T04:25:35.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:35.558 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:35.099+0000 7f7dad75b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T04:25:35.559 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:35.193+0000 7f7dad75b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T04:25:35.559 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:35.557+0000 7f7dad75b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-17T04:25:35.567 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:25:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z[37349]: 2024-05-17T04:25:35.162+0000 7f10e2136200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T04:25:35.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:35.755+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fb8e4012050 con 0x7fb8ec104620 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/19238223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: Standby manager daemon z started 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T04:25:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:35 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/19238223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: Standby manager daemon z started 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T04:25:36.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:35 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/19238223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:25:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: Standby manager daemon z started 2024-05-17T04:25:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-17T04:25:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:36.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-17T04:25:36.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:35 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.110:0/294881629' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:36.240 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:35.911+0000 7f7dad75b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T04:25:36.241 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.002+0000 7f7dad75b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T04:25:36.565 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.239+0000 7f7dad75b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T04:25:36.565 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.370+0000 7f7dad75b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T04:25:36.565 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.463+0000 7f7dad75b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T04:25:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:36 smithi105 ceph-mon[34795]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-17T04:25:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T04:25:37.065 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.785+0000 7f7dad75b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T04:25:37.065 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:36.878+0000 7f7dad75b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T04:25:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:36 smithi110 ceph-mon[35497]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-17T04:25:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T04:25:37.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:36 smithi050 ceph-mon[31739]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-17T04:25:37.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-17T04:25:37.766 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:37.484+0000 7f7dad75b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T04:25:37.766 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:25:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y[36168]: 2024-05-17T04:25:37.702+0000 7f7dad75b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T04:25:37.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:37.777+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mon.0 v2:172.21.15.50:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb8e4054b10 con 0x7fb8ec104620 2024-05-17T04:25:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: Standby manager daemon y started 2024-05-17T04:25:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T04:25:38.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:38.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T04:25:38.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:37 smithi105 ceph-mon[34795]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:38.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:38.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: Standby manager daemon y started 2024-05-17T04:25:38.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T04:25:38.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:38.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T04:25:38.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:37 smithi110 ceph-mon[35497]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:38.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:38.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: Standby manager daemon y started 2024-05-17T04:25:38.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-17T04:25:38.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-17T04:25:38.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-17T04:25:38.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:37 smithi050 ceph-mon[31739]: from='mgr.? 172.21.15.105:0/2733333595' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-17T04:25:38.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:38 smithi050 ceph-mon[31739]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T04:25:38.881 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T04:25:39.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:38 smithi105 ceph-mon[34795]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T04:25:39.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T04:25:39.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:38 smithi110 ceph-mon[35497]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-17T04:25:39.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-17T04:25:39.954 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:39 smithi050 ceph-mon[31739]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:39.954 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:40.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:39 smithi105 ceph-mon[34795]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:40.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:40.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:39 smithi110 ceph-mon[35497]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:40.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:41.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:41.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:41.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:41.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:41.920 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:41 smithi050 ceph-mon[31739]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:42.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:41 smithi105 ceph-mon[34795]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:41 smithi110 ceph-mon[35497]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:42.940 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T04:25:42.940 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:42.941 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:42 smithi050 ceph-mon[31739]: Deploying daemon osd.0 on smithi050 2024-05-17T04:25:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T04:25:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:42 smithi105 ceph-mon[34795]: Deploying daemon osd.0 on smithi050 2024-05-17T04:25:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-17T04:25:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:42 smithi110 ceph-mon[35497]: Deploying daemon osd.0 on smithi050 2024-05-17T04:25:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:43 smithi105 ceph-mon[34795]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:43 smithi110 ceph-mon[35497]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:43 smithi050 ceph-mon[31739]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:45 smithi105 ceph-mon[34795]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:45 smithi110 ceph-mon[35497]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:45 smithi050 ceph-mon[31739]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:47 smithi105 ceph-mon[34795]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:47 smithi110 ceph-mon[35497]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:47 smithi050 ceph-mon[31739]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:48.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:48.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:49.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:49 smithi050 ceph-mon[31739]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:49 smithi105 ceph-mon[34795]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:49 smithi110 ceph-mon[35497]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:51.228 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 0 on host 'smithi050' 2024-05-17T04:25:51.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.227+0000 7fb8e37fe700 1 -- 172.21.15.50:0/4207721356 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8ec0646a0 con 0x7fb8d803e750 2024-05-17T04:25:51.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 msgr2=0x7fb8d8040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:51.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 0x7fb8d8040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b650 tx=0x7fb8e801a040 comp rx=0 tx=0).stop 2024-05-17T04:25:51.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 msgr2=0x7fb8ec10bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:51.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ef30 tx=0x7fb8e400c3a0 comp rx=0 tx=0).stop 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 shutdown_connections 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8d803e750 0x7fb8d8040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8ec104620 0x7fb8ec10bf60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8ec1030f0 0x7fb8ec10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 --2- 172.21.15.50:0/4207721356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8ec103ad0 0x7fb8ec10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 >> 172.21.15.50:0/4207721356 conn(0x7fb8ec0fdb50 msgr2=0x7fb8ec1070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 shutdown_connections 2024-05-17T04:25:51.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:25:51.230+0000 7fb8f42f9700 1 -- 172.21.15.50:0/4207721356 wait complete. 2024-05-17T04:25:51.460 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.460 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.460 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.461 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:51.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:51.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:51.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:51.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:51.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:51.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:51.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:51.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:51.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:25:51.878 DEBUG:teuthology.orchestra.run.smithi050:osd.0> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.0.service 2024-05-17T04:25:51.881 INFO:tasks.cephadm:Deploying osd.1 on smithi105 with /dev/vg_nvme/lv_4... 2024-05-17T04:25:51.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T04:25:52.123 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:25:52.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:52 smithi105 ceph-mon[34795]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T04:25:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:52 smithi110 ceph-mon[35497]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T04:25:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:52 smithi050 ceph-mon[31739]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-17T04:25:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:53 smithi105 ceph-mon[34795]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T04:25:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:53 smithi105 ceph-mon[34795]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T04:25:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:53 smithi105 ceph-mon[34795]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:25:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:53 smithi105 ceph-mon[34795]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:53.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:53 smithi110 ceph-mon[35497]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T04:25:53.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:53 smithi110 ceph-mon[35497]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T04:25:53.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:53 smithi110 ceph-mon[35497]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:25:53.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:53.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:53 smithi110 ceph-mon[35497]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:53 smithi050 ceph-mon[31739]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-17T04:25:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:53 smithi050 ceph-mon[31739]: osdmap e6: 1 total, 0 up, 1 in 2024-05-17T04:25:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:53 smithi050 ceph-mon[31739]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:25:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:53 smithi050 ceph-mon[31739]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:53.935 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T04:25:53.935 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T04:25:53.935 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-05-17T04:25:53.935 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-05-17T04:25:53.935 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011567 s, 907 MB/s 2024-05-17T04:25:53.936 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-05-17T04:25:54.169 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:25:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0[44169]: 2024-05-17T04:25:53.858+0000 7fbe0789c700 -1 osd.0 0 waiting for initial osdmap 2024-05-17T04:25:54.169 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:25:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0[44169]: 2024-05-17T04:25:53.863+0000 7fbdfee48700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:25:54.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:54 smithi105 ceph-mon[34795]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:25:54.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:54 smithi105 ceph-mon[34795]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T04:25:54.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:54 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:54 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:54 smithi105 ceph-mon[34795]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' 2024-05-17T04:25:54.552 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-05-17T04:25:54.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:54 smithi050 ceph-mon[31739]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:25:54.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:54 smithi050 ceph-mon[31739]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T04:25:54.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:54 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:54 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:54 smithi050 ceph-mon[31739]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' 2024-05-17T04:25:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:54 smithi110 ceph-mon[35497]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:25:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:54 smithi110 ceph-mon[35497]: osdmap e7: 1 total, 0 up, 1 in 2024-05-17T04:25:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:54 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:54 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:54.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:54 smithi110 ceph-mon[35497]: from='osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330]' entity='osd.0' 2024-05-17T04:25:54.797 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:25:55.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:25:55.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:25:55.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] boot 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi050 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:55.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:25:55.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] boot 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi050 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:55.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:25:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:25:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: osd.0 [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] boot 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: osdmap e8: 1 total, 1 up, 1 in 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi050 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:55.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:25:55.578 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.439+0000 7f8f4ff25700 1 -- 172.21.15.105:0/557777645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48104660 msgr2=0x7f8f4810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.439+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/557777645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48104660 0x7f8f4810af10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b0a0 tx=0x7f8f4000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 -- 172.21.15.105:0/557777645 shutdown_connections 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/557777645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48104660 0x7f8f4810af10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/557777645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48102b90 0x7f8f48102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/557777645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48103cc0 0x7f8f481040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 -- 172.21.15.105:0/557777645 >> 172.21.15.105:0/557777645 conn(0x7f8f480fe3c0 msgr2=0x7f8f48100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 -- 172.21.15.105:0/557777645 shutdown_connections 2024-05-17T04:25:56.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 -- 172.21.15.105:0/557777645 wait complete. 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 Processor -- start 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 -- start start 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.440+0000 7f8f4ff25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48102b90 0x7f8f4819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 0x7f8f481a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810e4c0 con 0x7f8f48104660 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810e340 con 0x7f8f48103cc0 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810e640 con 0x7f8f48102b90 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57768/0 (socket says 172.21.15.105:57768) 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 -- 172.21.15.105:0/1304292755 learned_addr learned my addr 172.21.15.105:0/1304292755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4dcc1700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48102b90 0x7f8f4819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:56.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4e4c2700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 0x7f8f481a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 -- 172.21.15.105:0/1304292755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48102b90 msgr2=0x7f8f4819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48102b90 0x7f8f4819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 -- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 msgr2=0x7f8f481a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 0x7f8f481a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 -- 172.21.15.105:0/1304292755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f481a3f20 con 0x7f8f48103cc0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4e4c2700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 0x7f8f481a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4d4c0700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c920 tx=0x7f8f3800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007b00 con 0x7f8f48103cc0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f38004cc0 con 0x7f8f48103cc0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f481a4210 con 0x7f8f48103cc0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38005440 con 0x7f8f48103cc0 2024-05-17T04:25:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.441+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a4750 con 0x7f8f48103cc0 2024-05-17T04:25:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.442+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f48102fa0 con 0x7f8f48103cc0 2024-05-17T04:25:56.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.445+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f8f38004750 con 0x7f8f48103cc0 2024-05-17T04:25:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.446+0000 7f8f3effd700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 0x7f8f340b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:25:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.446+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f8f380e06f0 con 0x7f8f48103cc0 2024-05-17T04:25:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.446+0000 7f8f4dcc1700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 0x7f8f340b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:25:56.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.446+0000 7f8f4dcc1700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 0x7f8f340b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005950 tx=0x7f8f4400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:25:56.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.449+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f38014030 con 0x7f8f48103cc0 2024-05-17T04:25:56.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:25:56.657+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8f481a4e30 con 0x7f8f340b27f0 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:57.538 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T04:25:57.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:25:57.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:57.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:57.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:57.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:25:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: osdmap e9: 1 total, 1 up, 1 in 2024-05-17T04:25:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:25:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:25:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:25:57.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:25:57.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:00.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:00.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/895495688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]': finished 2024-05-17T04:26:00.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T04:26:00.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:25:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:00.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:00.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/895495688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]': finished 2024-05-17T04:26:00.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T04:26:00.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:25:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/895495688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]: dispatch 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481"}]': finished 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: osdmap e10: 2 total, 1 up, 2 in 2024-05-17T04:26:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:25:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:01.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:00 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/1525115940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:01.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:00 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/1525115940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:01.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:00 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/1525115940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:02.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:01 smithi105 ceph-mon[34795]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:02.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:01 smithi110 ceph-mon[35497]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:02.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:01 smithi050 ceph-mon[31739]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:03 smithi105 ceph-mon[34795]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:03 smithi110 ceph-mon[35497]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:04.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:03 smithi050 ceph-mon[31739]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:06.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:05 smithi105 ceph-mon[34795]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:06.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:05 smithi110 ceph-mon[35497]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:06.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:05 smithi050 ceph-mon[31739]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:06 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T04:26:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:06 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:06 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T04:26:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:06 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:07.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:06 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-17T04:26:07.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:06 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:08.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:07 smithi105 ceph-mon[34795]: Deploying daemon osd.1 on smithi105 2024-05-17T04:26:08.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:07 smithi105 ceph-mon[34795]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:08.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:07 smithi110 ceph-mon[35497]: Deploying daemon osd.1 on smithi105 2024-05-17T04:26:08.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:07 smithi110 ceph-mon[35497]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:07 smithi050 ceph-mon[31739]: Deploying daemon osd.1 on smithi105 2024-05-17T04:26:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:07 smithi050 ceph-mon[31739]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:10.047 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:09 smithi105 ceph-mon[34795]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:09 smithi110 ceph-mon[35497]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:09 smithi050 ceph-mon[31739]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:11.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:11 smithi105 ceph-mon[34795]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:12.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:11 smithi110 ceph-mon[35497]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:12.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:11 smithi050 ceph-mon[31739]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:12.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:12.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:12.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:13.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:13.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:13.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:13.932 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:13 smithi105 ceph-mon[34795]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:13 smithi110 ceph-mon[35497]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:13 smithi050 ceph-mon[31739]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:15.077 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 1 on host 'smithi105' 2024-05-17T04:26:15.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.075+0000 7f8f3effd700 1 -- 172.21.15.105:0/1304292755 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f481a4e30 con 0x7f8f340b27f0 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 msgr2=0x7f8f340b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 0x7f8f340b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005950 tx=0x7f8f4400a300 comp rx=0 tx=0).stop 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 msgr2=0x7f8f4819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800c920 tx=0x7f8f3800cc30 comp rx=0 tx=0).stop 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 shutdown_connections 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8f340b27f0 0x7f8f340b4d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f48104660 0x7f8f481a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:15.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48102b90 0x7f8f4819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:15.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 --2- 172.21.15.105:0/1304292755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f48103cc0 0x7f8f4819c900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:15.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 >> 172.21.15.105:0/1304292755 conn(0x7f8f480fe3c0 msgr2=0x7f8f480ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:26:15.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 shutdown_connections 2024-05-17T04:26:15.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:26:15.078+0000 7f8f4ff25700 1 -- 172.21.15.105:0/1304292755 wait complete. 2024-05-17T04:26:15.706 DEBUG:teuthology.orchestra.run.smithi105:osd.1> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.1.service 2024-05-17T04:26:15.708 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/vg_nvme/lv_4... 2024-05-17T04:26:15.708 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-17T04:26:15.953 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:26:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:16.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:16 smithi105 ceph-mon[34795]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:16.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:16 smithi110 ceph-mon[35497]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:16 smithi050 ceph-mon[31739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-17T04:26:17.184 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T04:26:17.184 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T04:26:17.184 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:17.184 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.184 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.185 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:17 smithi105 ceph-mon[34795]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:17.185 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:17 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1[39370]: 2024-05-17T04:26:17.118+0000 7f445d26e700 -1 osd.1 0 waiting for initial osdmap 2024-05-17T04:26:17.185 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:17 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1[39370]: 2024-05-17T04:26:17.121+0000 7f4459023700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:17 smithi110 ceph-mon[35497]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:17.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-17T04:26:17.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: osdmap e11: 2 total, 1 up, 2 in 2024-05-17T04:26:17.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:17.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: from='osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:26:17.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:17 smithi050 ceph-mon[31739]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00822842 s, 1.3 GB/s 2024-05-17T04:26:17.774 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-17T04:26:18.296 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:18 smithi105 ceph-mon[34795]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:26:18.296 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:18 smithi105 ceph-mon[34795]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T04:26:18.296 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.296 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:18 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.353 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:18 smithi110 ceph-mon[35497]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:26:18.353 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:18 smithi110 ceph-mon[35497]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T04:26:18.353 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.353 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:18 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.355 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-05-17T04:26:18.522 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:18 smithi050 ceph-mon[31739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:26:18.522 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:18 smithi050 ceph-mon[31739]: osdmap e12: 2 total, 1 up, 2 in 2024-05-17T04:26:18.522 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.522 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:18 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:18.596 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:26:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] boot 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi105 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 17088M 2024-05-17T04:26:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] boot 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi105 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 17088M 2024-05-17T04:26:19.511 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.512 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:26:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: osd.1 [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] boot 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: osdmap e13: 2 total, 2 up, 2 in 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi105 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 17088M 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:19.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:19.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:19.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:19.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:20.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 -- 172.21.15.110:0/63651098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 msgr2=0x7ff87c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 --2- 172.21.15.110:0/63651098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c102fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8640099c0 tx=0x7ff864009cd0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 -- 172.21.15.110:0/63651098 shutdown_connections 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 --2- 172.21.15.110:0/63651098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c104660 0x7ff87c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 --2- 172.21.15.110:0/63651098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c102fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 --2- 172.21.15.110:0/63651098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c103cc0 0x7ff87c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.149+0000 7ff881176700 1 -- 172.21.15.110:0/63651098 >> 172.21.15.110:0/63651098 conn(0x7ff87c0fe3c0 msgr2=0x7ff87c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.150+0000 7ff881176700 1 -- 172.21.15.110:0/63651098 shutdown_connections 2024-05-17T04:26:20.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.150+0000 7ff881176700 1 -- 172.21.15.110:0/63651098 wait complete. 2024-05-17T04:26:20.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.150+0000 7ff881176700 1 Processor -- start 2024-05-17T04:26:20.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 -- start start 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c103cc0 0x7ff87c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c104660 0x7ff87c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10f5b0 con 0x7ff87c103cc0 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10f430 con 0x7ff87c104660 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff881176700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c10f730 con 0x7ff87c102b90 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52778/0 (socket says 172.21.15.110:52778) 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 -- 172.21.15.110:0/1790750387 learned_addr learned my addr 172.21.15.110:0/1790750387 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87a59c700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c103cc0 0x7ff87c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87b59e700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c104660 0x7ff87c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 -- 172.21.15.110:0/1790750387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c104660 msgr2=0x7ff87c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c104660 0x7ff87c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 -- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c103cc0 msgr2=0x7ff87c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c103cc0 0x7ff87c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.151+0000 7ff87ad9d700 1 -- 172.21.15.110:0/1790750387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c1a1cc0 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff87ad9d700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff864009990 tx=0x7ff86400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864027070 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff86400bd30 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1a1f50 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864022d90 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.152+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1a22f0 con 0x7ff87c102b90 2024-05-17T04:26:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.153+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c102fa0 con 0x7ff87c102b90 2024-05-17T04:26:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.155+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff86402a070 con 0x7ff87c102b90 2024-05-17T04:26:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.155+0000 7ff873fff700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 0x7ff8680b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.155+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7ff8640e1d30 con 0x7ff87c102b90 2024-05-17T04:26:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.156+0000 7ff87a59c700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 0x7ff8680b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:20.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.156+0000 7ff87a59c700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 0x7ff8680b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff86c005950 tx=0x7ff86c005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:26:20.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.159+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff864013070 con 0x7ff87c102b90 2024-05-17T04:26:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:20 smithi110 ceph-mon[35497]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T04:26:20.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:20.371+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff87c064770 con 0x7ff8680b27f0 2024-05-17T04:26:20.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:20 smithi105 ceph-mon[34795]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T04:26:20.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:20 smithi050 ceph-mon[31739]: osdmap e14: 2 total, 2 up, 2 in 2024-05-17T04:26:21.247 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:21 smithi110 ceph-mon[35497]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:21.247 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:21 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:21.247 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:21 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:21.247 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:21 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:21.247 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:21 smithi110 ceph-mon[35497]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:21.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:21 smithi105 ceph-mon[34795]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:21.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:21 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:21.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:21 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:21.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:21 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:21.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:21 smithi105 ceph-mon[34795]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:21.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:21 smithi050 ceph-mon[31739]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:21.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:21.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:21.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:21 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:21.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:21 smithi050 ceph-mon[31739]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:24.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:24.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/2350695763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]': finished 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:23 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/3828897202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/2350695763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]': finished 2024-05-17T04:26:24.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T04:26:24.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:24.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:23 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/3828897202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/2350695763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]: dispatch 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac2135de-eec8-43a3-bbeb-49c09e2b60e6"}]': finished 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: osdmap e15: 3 total, 2 up, 3 in 2024-05-17T04:26:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:24.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:23 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/3828897202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:26.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:25 smithi105 ceph-mon[34795]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:26.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:25 smithi110 ceph-mon[35497]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:26.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:25 smithi050 ceph-mon[31739]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:27.970 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:27 smithi110 ceph-mon[35497]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:27 smithi105 ceph-mon[34795]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:28.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:27 smithi050 ceph-mon[31739]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:29 smithi105 ceph-mon[34795]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:29 smithi110 ceph-mon[35497]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:29 smithi050 ceph-mon[31739]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:30.800 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T04:26:30.800 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:30.800 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:30 smithi110 ceph-mon[35497]: Deploying daemon osd.2 on smithi110 2024-05-17T04:26:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T04:26:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:30 smithi105 ceph-mon[34795]: Deploying daemon osd.2 on smithi110 2024-05-17T04:26:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-17T04:26:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:30 smithi050 ceph-mon[31739]: Deploying daemon osd.2 on smithi110 2024-05-17T04:26:31.980 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:31 smithi110 ceph-mon[35497]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:31 smithi105 ceph-mon[34795]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:32.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:31 smithi050 ceph-mon[31739]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:33.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:33 smithi110 ceph-mon[35497]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:34.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:33 smithi105 ceph-mon[34795]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:33 smithi050 ceph-mon[31739]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:36.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:35 smithi110 ceph-mon[35497]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:36.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:36.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:36.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:36.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:35 smithi105 ceph-mon[34795]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:36.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:36.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:36.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:35 smithi050 ceph-mon[31739]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:36.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:37.959 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:37 smithi110 ceph-mon[35497]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:37 smithi105 ceph-mon[34795]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:38.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:37 smithi050 ceph-mon[31739]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:39.035 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.034+0000 7ff873fff700 1 -- 172.21.15.110:0/1790750387 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff87c064770 con 0x7ff8680b27f0 2024-05-17T04:26:39.035 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 msgr2=0x7ff8680b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 0x7ff8680b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff86c005950 tx=0x7ff86c005eb0 comp rx=0 tx=0).stop 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 msgr2=0x7ff87c19a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff864009990 tx=0x7ff86400f740 comp rx=0 tx=0).stop 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 shutdown_connections 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff87c104660 0x7ff87c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff87c102b90 0x7ff87c19a160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff8680b27f0 0x7ff8680b4d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:39.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 --2- 172.21.15.110:0/1790750387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff87c103cc0 0x7ff87c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 >> 172.21.15.110:0/1790750387 conn(0x7ff87c0fe3c0 msgr2=0x7ff87c106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:26:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 shutdown_connections 2024-05-17T04:26:39.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:26:39.038+0000 7ff881176700 1 -- 172.21.15.110:0/1790750387 wait complete. 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:39.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:39.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:39.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:26:39.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:39.668 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.2.service 2024-05-17T04:26:39.670 INFO:tasks.cephadm:Deploying osd.3 on smithi050 with /dev/vg_nvme/lv_3... 2024-05-17T04:26:39.670 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T04:26:39.930 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:26:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:40 smithi105 ceph-mon[34795]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:40 smithi105 ceph-mon[34795]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:40 smithi105 ceph-mon[34795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:40 smithi110 ceph-mon[35497]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:40 smithi110 ceph-mon[35497]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:40 smithi110 ceph-mon[35497]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:40 smithi050 ceph-mon[31739]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:40 smithi050 ceph-mon[31739]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:40 smithi050 ceph-mon[31739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-17T04:26:41.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:41 smithi050 ceph-mon[31739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T04:26:41.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:41 smithi050 ceph-mon[31739]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T04:26:41.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:41.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:41 smithi050 ceph-mon[31739]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:41 smithi050 ceph-mon[31739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:41 smithi105 ceph-mon[34795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T04:26:41.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:41 smithi105 ceph-mon[34795]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T04:26:41.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:41.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:41 smithi105 ceph-mon[34795]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:41 smithi105 ceph-mon[34795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:41 smithi110 ceph-mon[35497]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-17T04:26:41.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:41 smithi110 ceph-mon[35497]: osdmap e16: 3 total, 2 up, 3 in 2024-05-17T04:26:41.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:41.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:41 smithi110 ceph-mon[35497]: from='osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:41 smithi110 ceph-mon[35497]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:26:41.318 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2[40638]: 2024-05-17T04:26:41.067+0000 7f46d7a66700 -1 osd.2 0 waiting for initial osdmap 2024-05-17T04:26:41.318 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2[40638]: 2024-05-17T04:26:41.093+0000 7f46d381b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:26:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T04:26:41.848 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T04:26:41.848 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-17T04:26:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-17T04:26:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834537 s, 1.3 GB/s 2024-05-17T04:26:41.849 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 2024-05-17T04:26:41.849 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-17T04:26:42.180 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:42 smithi050 ceph-mon[31739]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:42.180 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:42 smithi050 ceph-mon[31739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:26:42.180 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:42 smithi050 ceph-mon[31739]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T04:26:42.180 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.180 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:42 smithi110 ceph-mon[35497]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:42.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:42 smithi110 ceph-mon[35497]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:26:42.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:42 smithi110 ceph-mon[35497]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T04:26:42.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.222 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-05-17T04:26:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:42 smithi105 ceph-mon[34795]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-17T04:26:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:42 smithi105 ceph-mon[34795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:26:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:42 smithi105 ceph-mon[34795]: osdmap e17: 3 total, 2 up, 3 in 2024-05-17T04:26:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:42.818 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:26:43.201 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:26:43.201 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:26:43.201 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] boot 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:26:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] boot 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:43.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:43.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:26:43.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:26:43.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: osd.2 [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] boot 2024-05-17T04:26:43.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: osdmap e18: 3 total, 3 up, 3 in 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:26:43.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:43.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi110 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T04:26:44.299 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:44.299 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48539]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48539]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48539]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48542]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.300 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48542]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi110 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T04:26:44.371 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:44.371 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44633]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44633]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44636]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.372 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44636]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.409 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi110 2024-05-17T04:26:44.409 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-17T04:26:44.409 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: pgmap v80: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:44.410 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:44.410 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: osdmap e19: 3 total, 3 up, 3 in 2024-05-17T04:26:44.410 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43154]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43154]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43154]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.410 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.411 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.411 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43157]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 -- 172.21.15.50:0/1036501877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc102b90 msgr2=0x7ffbbc102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/1036501877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc102b90 0x7ffbbc102fa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80099c0 tx=0x7ffbb8009cd0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 -- 172.21.15.50:0/1036501877 shutdown_connections 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/1036501877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc104660 0x7ffbbc10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/1036501877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc102b90 0x7ffbbc102fa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/1036501877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 -- 172.21.15.50:0/1036501877 >> 172.21.15.50:0/1036501877 conn(0x7ffbbc0fe3c0 msgr2=0x7ffbbc100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:26:44.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 -- 172.21.15.50:0/1036501877 shutdown_connections 2024-05-17T04:26:44.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.551+0000 7ffbc46ff700 1 -- 172.21.15.50:0/1036501877 wait complete. 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48550]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48550]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48553]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.555 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48553]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.556 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:44 smithi050 sudo[48553]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.552+0000 7ffbc46ff700 1 Processor -- start 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.552+0000 7ffbc46ff700 1 -- start start 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc102b90 0x7ffbbc1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc104660 0x7ffbbc1a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10e490 con 0x7ffbbc103cc0 2024-05-17T04:26:44.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10e310 con 0x7ffbbc102b90 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc46ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10e610 con 0x7ffbbc104660 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40014/0 (socket says 172.21.15.50:40014) 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 -- 172.21.15.50:0/413841078 learned_addr learned my addr 172.21.15.50:0/413841078 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc249b700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc102b90 0x7ffbbc1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc2c9c700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc104660 0x7ffbbc1a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 -- 172.21.15.50:0/413841078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc104660 msgr2=0x7ffbbc1a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc104660 0x7ffbbc1a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 -- 172.21.15.50:0/413841078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc102b90 msgr2=0x7ffbbc1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:26:44.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc102b90 0x7ffbbc1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 -- 172.21.15.50:0/413841078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc1a8380 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.553+0000 7ffbc1c9a700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d950 tx=0x7ffbac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.554+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac016070 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.554+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc1a8670 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.554+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbac00fd20 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.554+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00e3b0 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.554+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc1a8bb0 con 0x7ffbbc103cc0 2024-05-17T04:26:44.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.555+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbbc102fa0 con 0x7ffbbc103cc0 2024-05-17T04:26:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.557+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ffbac019070 con 0x7ffbbc103cc0 2024-05-17T04:26:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.557+0000 7ffbb37fe700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 0x7ffba80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:26:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.558+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7ffbac0e0780 con 0x7ffbbc103cc0 2024-05-17T04:26:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.558+0000 7ffbc249b700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 0x7ffba80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:26:44.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.558+0000 7ffbc249b700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 0x7ffba80b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc1a19f0 tx=0x7ffbb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:26:44.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.561+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffbac0a9950 con 0x7ffbbc103cc0 2024-05-17T04:26:44.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:44.786+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffbbc1a92f0 con 0x7ffba80b27f0 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43160]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43160]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43163]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:44 smithi105 sudo[43163]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-17T04:26:44.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44639]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44639]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:44.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44642]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-17T04:26:44.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44642]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-17T04:26:44.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44642]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-17T04:26:44.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:44 smithi110 sudo[44642]: pam_unix(sudo:session): session closed for user root 2024-05-17T04:26:45.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:26:45.335+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffbac0a54e0 con 0x7ffbbc103cc0 2024-05-17T04:26:45.337 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.338 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:45.339 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:45.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:45.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T04:26:45.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:26:45.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: osdmap e20: 3 total, 3 up, 3 in 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:26:45.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:46.355 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:46 smithi050 ceph-mon[31739]: pgmap v83: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:46.355 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:46 smithi050 ceph-mon[31739]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:46.355 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:46 smithi050 ceph-mon[31739]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T04:26:46.355 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:46 smithi050 ceph-mon[31739]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T04:26:46.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:46 smithi105 ceph-mon[34795]: pgmap v83: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:46.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:46 smithi105 ceph-mon[34795]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:46.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:46 smithi105 ceph-mon[34795]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T04:26:46.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:46 smithi105 ceph-mon[34795]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T04:26:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:46 smithi110 ceph-mon[35497]: pgmap v83: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:46 smithi110 ceph-mon[35497]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:26:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:46 smithi110 ceph-mon[35497]: osdmap e21: 3 total, 3 up, 3 in 2024-05-17T04:26:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:46 smithi110 ceph-mon[35497]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-17T04:26:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:48 smithi050 ceph-mon[31739]: pgmap v85: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:48 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]: dispatch 2024-05-17T04:26:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:48 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]': finished 2024-05-17T04:26:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:48 smithi050 ceph-mon[31739]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T04:26:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:48 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:26:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:48 smithi105 ceph-mon[34795]: pgmap v85: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:48 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]: dispatch 2024-05-17T04:26:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:48 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]': finished 2024-05-17T04:26:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:48 smithi105 ceph-mon[34795]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T04:26:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:48 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:26:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:48 smithi110 ceph-mon[35497]: pgmap v85: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:48 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]: dispatch 2024-05-17T04:26:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:48 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3467956238' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5004e0dd-7a4c-4229-abfe-72186a1ebf29"}]': finished 2024-05-17T04:26:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:48 smithi110 ceph-mon[35497]: osdmap e22: 4 total, 3 up, 4 in 2024-05-17T04:26:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:48 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:26:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3741559910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:49.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3741559910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3741559910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:26:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:50 smithi105 ceph-mon[34795]: pgmap v87: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:50 smithi110 ceph-mon[35497]: pgmap v87: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:50 smithi050 ceph-mon[31739]: pgmap v87: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:26:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:52 smithi050 ceph-mon[31739]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-17T04:26:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:52 smithi105 ceph-mon[34795]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-17T04:26:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:52 smithi110 ceph-mon[35497]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-17T04:26:54.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:54 smithi050 ceph-mon[31739]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-17T04:26:54.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:54 smithi105 ceph-mon[34795]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-17T04:26:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:54 smithi110 ceph-mon[35497]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-17T04:26:55.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T04:26:55.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:55.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T04:26:55.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-17T04:26:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:26:56.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:56 smithi050 ceph-mon[31739]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 191 KiB/s wr, 12 op/s 2024-05-17T04:26:56.268 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:56 smithi050 ceph-mon[31739]: Deploying daemon osd.3 on smithi050 2024-05-17T04:26:56.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:56 smithi105 ceph-mon[34795]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 191 KiB/s wr, 12 op/s 2024-05-17T04:26:56.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:56 smithi105 ceph-mon[34795]: Deploying daemon osd.3 on smithi050 2024-05-17T04:26:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:56 smithi110 ceph-mon[35497]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 191 KiB/s wr, 12 op/s 2024-05-17T04:26:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:56 smithi110 ceph-mon[35497]: Deploying daemon osd.3 on smithi050 2024-05-17T04:26:58.493 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:26:58 smithi050 ceph-mon[31739]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-17T04:26:58.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:26:58 smithi105 ceph-mon[34795]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-17T04:26:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:26:58 smithi110 ceph-mon[35497]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-17T04:27:00.177 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:00 smithi050 ceph-mon[31739]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 165 KiB/s wr, 11 op/s 2024-05-17T04:27:00.367 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:00 smithi105 ceph-mon[34795]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 165 KiB/s wr, 11 op/s 2024-05-17T04:27:00.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:00 smithi110 ceph-mon[35497]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 165 KiB/s wr, 11 op/s 2024-05-17T04:27:02.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:01 smithi105 ceph-mon[34795]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-17T04:27:02.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:02.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:01 smithi110 ceph-mon[35497]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-17T04:27:02.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:02.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:01 smithi050 ceph-mon[31739]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-17T04:27:02.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:02.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:04.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:03 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:04.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:03 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:04.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:04.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:04.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:03 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.695+0000 7ffbb37fe700 1 -- 172.21.15.50:0/413841078 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffbbc1a92f0 con 0x7ffba80b27f0 2024-05-17T04:27:04.696 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 3 on host 'smithi050' 2024-05-17T04:27:04.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 msgr2=0x7ffba80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:04.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 0x7ffba80b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc1a19f0 tx=0x7ffbb8005fb0 comp rx=0 tx=0).stop 2024-05-17T04:27:04.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 msgr2=0x7ffbbc1a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:04.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d950 tx=0x7ffbac00dc60 comp rx=0 tx=0).stop 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 shutdown_connections 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbbc104660 0x7ffbbc1a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbbc102b90 0x7ffbbc1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffba80b27f0 0x7ffba80b4d00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 --2- 172.21.15.50:0/413841078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffbbc103cc0 0x7ffbbc1a0f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 >> 172.21.15.50:0/413841078 conn(0x7ffbbc0fe3c0 msgr2=0x7ffbbc0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 shutdown_connections 2024-05-17T04:27:04.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:27:04.698+0000 7ffbc46ff700 1 -- 172.21.15.50:0/413841078 wait complete. 2024-05-17T04:27:05.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:04 smithi105 ceph-mon[34795]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T04:27:05.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:05.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:04 smithi110 ceph-mon[35497]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T04:27:05.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:05.070 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:04 smithi050 ceph-mon[31739]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-17T04:27:05.070 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.070 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:05.070 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:05.390 DEBUG:teuthology.orchestra.run.smithi050:osd.3> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.3.service 2024-05-17T04:27:05.393 INFO:tasks.cephadm:Deploying osd.4 on smithi105 with /dev/vg_nvme/lv_3... 2024-05-17T04:27:05.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T04:27:05.638 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:27:05.974 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:05 smithi050 ceph-mon[31739]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:05.975 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:05 smithi050 ceph-mon[31739]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T04:27:05.975 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:05 smithi050 ceph-mon[31739]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T04:27:05.975 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:05 smithi050 ceph-mon[31739]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:27:05.975 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:05.975 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:27:05 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:27:05.766+0000 7f24c8bce700 -1 osd.3 0 waiting for initial osdmap 2024-05-17T04:27:05.975 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:27:05 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:27:05.773+0000 7f24c4983700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:27:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:05 smithi105 ceph-mon[34795]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:05 smithi105 ceph-mon[34795]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T04:27:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:05 smithi105 ceph-mon[34795]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T04:27:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:05 smithi105 ceph-mon[34795]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:27:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:06.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:05 smithi110 ceph-mon[35497]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:06.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:05 smithi110 ceph-mon[35497]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-17T04:27:06.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:05 smithi110 ceph-mon[35497]: osdmap e23: 4 total, 3 up, 4 in 2024-05-17T04:27:06.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:05 smithi110 ceph-mon[35497]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:27:06.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:06 smithi050 ceph-mon[31739]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:27:07.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:06 smithi050 ceph-mon[31739]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T04:27:07.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:06 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.058 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:06 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:06 smithi105 ceph-mon[34795]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:27:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:06 smithi105 ceph-mon[34795]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T04:27:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:06 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:06 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:06 smithi110 ceph-mon[35497]: from='osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:27:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:06 smithi110 ceph-mon[35497]: osdmap e24: 4 total, 3 up, 4 in 2024-05-17T04:27:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:06 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:06 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.647 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T04:27:07.647 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T04:27:07.647 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-05-17T04:27:07.647 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-05-17T04:27:07.647 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113034 s, 928 MB/s 2024-05-17T04:27:07.648 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 2024-05-17T04:27:07.648 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-05-17T04:27:07.967 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] boot 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T04:27:07.968 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] boot 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T04:27:07.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:08.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:27:08.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:27:08.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-17T04:27:08.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:08.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: osd.3 [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] boot 2024-05-17T04:27:08.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: osdmap e25: 4 total, 4 up, 4 in 2024-05-17T04:27:08.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-17T04:27:08.318 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-05-17T04:27:08.581 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:27:08.870 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:08.871 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:09.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: osdmap e26: 4 total, 4 up, 4 in 2024-05-17T04:27:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:09.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:09.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:09.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:09.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:09.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:10.016 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:09 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi050 2024-05-17T04:27:10.016 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:09 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-17T04:27:10.016 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:09 smithi105 ceph-mon[34795]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:10.016 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:09 smithi105 ceph-mon[34795]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T04:27:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:09 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi050 2024-05-17T04:27:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:09 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-17T04:27:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:09 smithi110 ceph-mon[35497]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:09 smithi110 ceph-mon[35497]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T04:27:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:09 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi050 2024-05-17T04:27:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:09 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-17T04:27:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:09 smithi050 ceph-mon[31739]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:09 smithi050 ceph-mon[31739]: osdmap e27: 4 total, 4 up, 4 in 2024-05-17T04:27:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 -- 172.21.15.105:0/884537654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0109950 msgr2=0x7f72a010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 --2- 172.21.15.105:0/884537654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0109950 0x7f72a010be60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f729800b0a0 tx=0x7f729800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 -- 172.21.15.105:0/884537654 shutdown_connections 2024-05-17T04:27:10.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 --2- 172.21.15.105:0/884537654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0109950 0x7f72a010be60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 --2- 172.21.15.105:0/884537654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 0x7f72a0106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 --2- 172.21.15.105:0/884537654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0106df0 0x7f72a0109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 -- 172.21.15.105:0/884537654 >> 172.21.15.105:0/884537654 conn(0x7f72a00fe3c0 msgr2=0x7f72a0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 -- 172.21.15.105:0/884537654 shutdown_connections 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 -- 172.21.15.105:0/884537654 wait complete. 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.252+0000 7f72a646e700 1 Processor -- start 2024-05-17T04:27:10.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 -- start start 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 0x7f72a019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 0x7f72a01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010f590 con 0x7f72a0109950 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010f410 con 0x7f72a0106df0 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a646e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010f710 con 0x7f72a006cce0 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42958/0 (socket says 172.21.15.105:42958) 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 -- 172.21.15.105:0/2669508404 learned_addr learned my addr 172.21.15.105:0/2669508404 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729ffff700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 0x7f72a019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a4a0b700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 0x7f72a01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 -- 172.21.15.105:0/2669508404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 msgr2=0x7f72a019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:10.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 0x7f72a019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 -- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 msgr2=0x7f72a01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 0x7f72a01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f729f7fe700 1 -- 172.21.15.105:0/2669508404 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a01a1c50 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.253+0000 7f72a4a0b700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 0x7f72a01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f729f7fe700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9a0 tx=0x7f729000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729000cbd0 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a01a1f40 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f729000cd70 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290018680 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.254+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a23f0 con 0x7f72a0106df0 2024-05-17T04:27:10.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.255+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0107280 con 0x7f72a0106df0 2024-05-17T04:27:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.256+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7290010430 con 0x7f72a0106df0 2024-05-17T04:27:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.257+0000 7f729d7fa700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 0x7f728c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.257+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f72900e14f0 con 0x7f72a0106df0 2024-05-17T04:27:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.257+0000 7f729ffff700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 0x7f728c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.258+0000 7f729ffff700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 0x7f728c0b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a006c9e0 tx=0x7f7288007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:27:10.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.260+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72900aa430 con 0x7f72a0106df0 2024-05-17T04:27:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:10.475+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f72a00646b0 con 0x7f728c0b2880 2024-05-17T04:27:11.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:11.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:11.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:11.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:11.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:11.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:11.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:11.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:11.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:12.061 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:11 smithi105 ceph-mon[34795]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:12.061 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:11 smithi105 ceph-mon[34795]: pgmap v103: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:12.061 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:11 smithi105 ceph-mon[34795]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-17T04:27:12.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:11 smithi110 ceph-mon[35497]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:12.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:11 smithi110 ceph-mon[35497]: pgmap v103: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:12.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:11 smithi110 ceph-mon[35497]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-17T04:27:12.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:11 smithi050 ceph-mon[31739]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:12.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:11 smithi050 ceph-mon[31739]: pgmap v103: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:12.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:11 smithi050 ceph-mon[31739]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-17T04:27:14.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:14.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/4077173220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]': finished 2024-05-17T04:27:14.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T04:27:14.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/4077173220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]': finished 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T04:27:14.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/4077173220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]: dispatch 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2af1d73f-56c3-481a-b9ca-26dd81248baf"}]': finished 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: osdmap e28: 5 total, 4 up, 5 in 2024-05-17T04:27:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:15.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:14 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/113723259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:15.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:14 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/113723259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:15.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:14 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/113723259' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:15 smithi105 ceph-mon[34795]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T04:27:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:15 smithi105 ceph-mon[34795]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-17T04:27:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:15 smithi105 ceph-mon[34795]: Cluster is now healthy 2024-05-17T04:27:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:15 smithi110 ceph-mon[35497]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T04:27:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:15 smithi110 ceph-mon[35497]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-17T04:27:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:15 smithi110 ceph-mon[35497]: Cluster is now healthy 2024-05-17T04:27:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:15 smithi050 ceph-mon[31739]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-17T04:27:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:15 smithi050 ceph-mon[31739]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-17T04:27:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:15 smithi050 ceph-mon[31739]: Cluster is now healthy 2024-05-17T04:27:18.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:17 smithi105 ceph-mon[34795]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:27:18.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:17 smithi110 ceph-mon[35497]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:27:18.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:17 smithi050 ceph-mon[31739]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:27:19.949 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:19 smithi105 ceph-mon[34795]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:19 smithi110 ceph-mon[35497]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:20.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:19 smithi050 ceph-mon[31739]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:21.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T04:27:21.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:21.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T04:27:21.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:21.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-17T04:27:21.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:22.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:21 smithi105 ceph-mon[34795]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:22.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:21 smithi105 ceph-mon[34795]: Deploying daemon osd.4 on smithi105 2024-05-17T04:27:22.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:21 smithi110 ceph-mon[35497]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:22.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:21 smithi110 ceph-mon[35497]: Deploying daemon osd.4 on smithi105 2024-05-17T04:27:22.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:21 smithi050 ceph-mon[31739]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:22.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:21 smithi050 ceph-mon[31739]: Deploying daemon osd.4 on smithi105 2024-05-17T04:27:23.933 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:23 smithi105 ceph-mon[34795]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:24.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:23 smithi110 ceph-mon[35497]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:24.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:23 smithi050 ceph-mon[31739]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:27:26.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:25 smithi105 ceph-mon[34795]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-17T04:27:26.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:25 smithi050 ceph-mon[31739]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-17T04:27:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:25 smithi110 ceph-mon[35497]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-05-17T04:27:27.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:27 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:27 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:27 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:27.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:27 smithi105 ceph-mon[34795]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:27.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:27 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:27 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:27 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:27.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:27 smithi110 ceph-mon[35497]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:27 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:27 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:27 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:27 smithi050 ceph-mon[31739]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:29.953 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:30.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:30.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:30.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:30.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.296+0000 7f729d7fa700 1 -- 172.21.15.105:0/2669508404 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72a00646b0 con 0x7f728c0b2880 2024-05-17T04:27:30.297 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 4 on host 'smithi105' 2024-05-17T04:27:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 msgr2=0x7f728c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 0x7f728c0b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a006c9e0 tx=0x7f7288007400 comp rx=0 tx=0).stop 2024-05-17T04:27:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 msgr2=0x7f72a019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:30.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9a0 tx=0x7f729000ecb0 comp rx=0 tx=0).stop 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 shutdown_connections 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f728c0b2880 0x7f728c0b4d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72a0109950 0x7f72a01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72a006cce0 0x7f72a019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 --2- 172.21.15.105:0/2669508404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72a0106df0 0x7f72a019a630 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 >> 172.21.15.105:0/2669508404 conn(0x7f72a00fe3c0 msgr2=0x7f72a0104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 shutdown_connections 2024-05-17T04:27:30.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:27:30.299+0000 7f72a646e700 1 -- 172.21.15.105:0/2669508404 wait complete. 2024-05-17T04:27:30.975 DEBUG:teuthology.orchestra.run.smithi105:osd.4> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.4.service 2024-05-17T04:27:30.977 INFO:tasks.cephadm:Deploying osd.5 on smithi110 with /dev/vg_nvme/lv_3... 2024-05-17T04:27:30.977 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-17T04:27:31.215 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:27:31.539 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.540 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.540 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.540 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.540 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:31.540 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:31 smithi110 ceph-mon[35497]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:31.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:31 smithi105 ceph-mon[34795]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:31.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:31.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-17T04:27:31.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:31.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:31 smithi050 ceph-mon[31739]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:32.455 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:32 smithi105 ceph-mon[34795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T04:27:32.455 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:32 smithi105 ceph-mon[34795]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T04:27:32.455 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:32.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:32 smithi105 ceph-mon[34795]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:32.456 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:32 smithi105 ceph-mon[34795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:32.456 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:27:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:27:32.308+0000 7f48fe472700 -1 osd.4 0 waiting for initial osdmap 2024-05-17T04:27:32.456 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:27:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:27:32.317+0000 7f48fa227700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:27:32.522 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:32 smithi110 ceph-mon[35497]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T04:27:32.523 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:32 smithi110 ceph-mon[35497]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T04:27:32.523 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:32.523 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:32 smithi110 ceph-mon[35497]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:32.523 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:32 smithi110 ceph-mon[35497]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:32 smithi050 ceph-mon[31739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-17T04:27:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:32 smithi050 ceph-mon[31739]: osdmap e29: 5 total, 4 up, 5 in 2024-05-17T04:27:32.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:32.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:32 smithi050 ceph-mon[31739]: from='osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:32.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:32 smithi050 ceph-mon[31739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:27:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-17T04:27:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-17T04:27:33.066 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-17T04:27:33.066 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-17T04:27:33.067 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00820763 s, 1.3 GB/s 2024-05-17T04:27:33.067 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.447 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:33 smithi110 ceph-mon[35497]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:33.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:27:33.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:27:33.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:27:33.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T04:27:33.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:33 smithi050 ceph-mon[31739]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:33.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:27:33.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:27:33.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:27:33.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: osdmap e30: 5 total, 4 up, 5 in 2024-05-17T04:27:33.682 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.682 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:33.682 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:33 smithi105 ceph-mon[34795]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-17T04:27:33.685 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-05-17T04:27:33.921 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:27:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] boot 2024-05-17T04:27:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi105 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 8544M 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:34.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] boot 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi105 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 8544M 2024-05-17T04:27:34.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.716 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:34.716 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:34.716 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: osd.4 [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] boot 2024-05-17T04:27:34.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: osdmap e31: 5 total, 5 up, 5 in 2024-05-17T04:27:34.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi105 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 8544M 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:34.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:35.369 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:35 smithi110 ceph-mon[35497]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T04:27:35.369 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:35 smithi110 ceph-mon[35497]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 -- 172.21.15.110:0/2465431201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4107150 msgr2=0x7f03f4109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 --2- 172.21.15.110:0/2465431201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4107150 0x7f03f4109570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009990 tx=0x7f03e0009ca0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 -- 172.21.15.110:0/2465431201 shutdown_connections 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 --2- 172.21.15.110:0/2465431201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 --2- 172.21.15.110:0/2465431201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4102020 0x7f03f4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 --2- 172.21.15.110:0/2465431201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4107150 0x7f03f4109570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 -- 172.21.15.110:0/2465431201 >> 172.21.15.110:0/2465431201 conn(0x7f03f40fdb50 msgr2=0x7f03f40fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 -- 172.21.15.110:0/2465431201 shutdown_connections 2024-05-17T04:27:35.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.409+0000 7f03fab2c700 1 -- 172.21.15.110:0/2465431201 wait complete. 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 Processor -- start 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 -- start start 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 0x7f03f419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f41a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410f610 con 0x7f03f4107150 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410f490 con 0x7f03f4109ab0 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03fab2c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03f410f790 con 0x7f03f4102020 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44024/0 (socket says 172.21.15.110:44024) 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 -- 172.21.15.110:0/3681810807 learned_addr learned my addr 172.21.15.110:0/3681810807 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:27:35.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f3fff700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 0x7f03f419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f90c9700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f41a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 -- 172.21.15.110:0/3681810807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 msgr2=0x7f03f41a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f41a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 -- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 msgr2=0x7f03f419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 0x7f03f419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f88c8700 1 -- 172.21.15.110:0/3681810807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f41a1d50 con 0x7f03f4102020 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f90c9700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.410+0000 7f03f3fff700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 0x7f03f419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03f88c8700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f03e800ada0 tx=0x7f03e800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8017020 con 0x7f03f4102020 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41a1fe0 con 0x7f03f4102020 2024-05-17T04:27:35.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e800ce10 con 0x7f03f4102020 2024-05-17T04:27:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8005440 con 0x7f03f4102020 2024-05-17T04:27:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.411+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a2590 con 0x7f03f4102020 2024-05-17T04:27:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.412+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f4102430 con 0x7f03f4102020 2024-05-17T04:27:35.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.413+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f03e8004ad0 con 0x7f03f4102020 2024-05-17T04:27:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.414+0000 7f03f1ffb700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 0x7f03e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:27:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.414+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f03e80e0950 con 0x7f03f4102020 2024-05-17T04:27:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.414+0000 7f03f3fff700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 0x7f03e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:27:35.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.415+0000 7f03f3fff700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 0x7f03e40b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:27:35.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.419+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03e8021020 con 0x7f03f4102020 2024-05-17T04:27:35.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:35 smithi050 ceph-mon[31739]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T04:27:35.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:35 smithi050 ceph-mon[31739]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:35.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:35.632+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f03f4064760 con 0x7f03e40b2880 2024-05-17T04:27:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:35 smithi105 ceph-mon[34795]: osdmap e32: 5 total, 5 up, 5 in 2024-05-17T04:27:35.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:35 smithi105 ceph-mon[34795]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:36.538 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:36 smithi110 ceph-mon[35497]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:36.538 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:36.539 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:36.539 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:36.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:36 smithi105 ceph-mon[34795]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:36.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:36.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:36.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:36.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:36 smithi050 ceph-mon[31739]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:27:36.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:27:36.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:27:36.827 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:37.536 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:37 smithi110 ceph-mon[35497]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:37.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:37 smithi105 ceph-mon[34795]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:37 smithi050 ceph-mon[31739]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:38.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:38 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/2756054950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:38 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:38 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]': finished 2024-05-17T04:27:38.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:38 smithi110 ceph-mon[35497]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T04:27:38.776 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:38.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:38 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/2756054950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:38 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:38 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]': finished 2024-05-17T04:27:38.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:38 smithi105 ceph-mon[34795]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T04:27:38.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:38.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:38 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/2756054950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:38 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]: dispatch 2024-05-17T04:27:38.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:38 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fec3a3b-bf8f-4065-8427-66d05c1a81a5"}]': finished 2024-05-17T04:27:38.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:38 smithi050 ceph-mon[31739]: osdmap e33: 6 total, 5 up, 6 in 2024-05-17T04:27:38.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:39.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:39 smithi105 ceph-mon[34795]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:39.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:39 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/2913504181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:39.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:39 smithi110 ceph-mon[35497]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:39.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:39 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/2913504181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:39.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:39 smithi050 ceph-mon[31739]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:39.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:39 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/2913504181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:27:42.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:41 smithi105 ceph-mon[34795]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:41 smithi110 ceph-mon[35497]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:42.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:41 smithi050 ceph-mon[31739]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:43.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:43 smithi110 ceph-mon[35497]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:43 smithi105 ceph-mon[34795]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:43 smithi050 ceph-mon[31739]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:45 smithi105 ceph-mon[34795]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T04:27:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:45 smithi110 ceph-mon[35497]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T04:27:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:45 smithi050 ceph-mon[31739]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-17T04:27:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:46.972 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:46 smithi110 ceph-mon[35497]: Deploying daemon osd.5 on smithi110 2024-05-17T04:27:47.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:46 smithi105 ceph-mon[34795]: Deploying daemon osd.5 on smithi110 2024-05-17T04:27:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:46 smithi050 ceph-mon[31739]: Deploying daemon osd.5 on smithi110 2024-05-17T04:27:47.972 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:47 smithi110 ceph-mon[35497]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:47.977 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:47 smithi050 ceph-mon[31739]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:47 smithi105 ceph-mon[34795]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:49 smithi105 ceph-mon[34795]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:49 smithi110 ceph-mon[35497]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:50.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:49 smithi050 ceph-mon[31739]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:52.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:51 smithi110 ceph-mon[35497]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:52.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:52.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:51 smithi105 ceph-mon[34795]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:52.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:51 smithi050 ceph-mon[31739]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:54.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:53 smithi105 ceph-mon[34795]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:54.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:53 smithi110 ceph-mon[35497]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:54.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:53 smithi050 ceph-mon[31739]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:55.002 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 5 on host 'smithi110' 2024-05-17T04:27:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.001+0000 7f03f1ffb700 1 -- 172.21.15.110:0/3681810807 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03f4064760 con 0x7f03e40b2880 2024-05-17T04:27:55.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 msgr2=0x7f03e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:55.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 0x7f03e40b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e0005fb0 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 msgr2=0x7f03f419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f03e800ada0 tx=0x7f03e800c330 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 shutdown_connections 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f4109ab0 0x7f03f41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f4102020 0x7f03f419a1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03e40b2880 0x7f03e40b4d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 --2- 172.21.15.110:0/3681810807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03f4107150 0x7f03f419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 >> 172.21.15.110:0/3681810807 conn(0x7f03f40fdb50 msgr2=0x7f03f4108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 shutdown_connections 2024-05-17T04:27:55.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:27:55.004+0000 7f03fab2c700 1 -- 172.21.15.110:0/3681810807 wait complete. 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:55.672 DEBUG:teuthology.orchestra.run.smithi110:osd.5> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.5.service 2024-05-17T04:27:55.675 INFO:tasks.cephadm:Deploying osd.6 on smithi050 with /dev/vg_nvme/lv_2... 2024-05-17T04:27:55.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T04:27:55.932 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:27:56.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:56 smithi110 ceph-mon[35497]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:56.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:56.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:56 smithi110 ceph-mon[35497]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:56.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:56 smithi110 ceph-mon[35497]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:56 smithi105 ceph-mon[34795]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:56 smithi105 ceph-mon[34795]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:56 smithi105 ceph-mon[34795]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:56 smithi050 ceph-mon[31739]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:27:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:56 smithi050 ceph-mon[31739]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:56.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:56 smithi050 ceph-mon[31739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-17T04:27:57.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:57 smithi105 ceph-mon[34795]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T04:27:57.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:57 smithi105 ceph-mon[34795]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T04:27:57.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:57.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:57 smithi105 ceph-mon[34795]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:57 smithi105 ceph-mon[34795]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:57 smithi110 ceph-mon[35497]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T04:27:57.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:57 smithi110 ceph-mon[35497]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T04:27:57.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:57.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:57 smithi110 ceph-mon[35497]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:57 smithi110 ceph-mon[35497]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:57 smithi050 ceph-mon[31739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-17T04:27:57.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:57 smithi050 ceph-mon[31739]: osdmap e34: 6 total, 5 up, 6 in 2024-05-17T04:27:57.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:57.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:57 smithi050 ceph-mon[31739]: from='osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:57 smithi050 ceph-mon[31739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:27:57.818 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:27:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:27:57.548+0000 7f782fd84700 -1 osd.5 0 waiting for initial osdmap 2024-05-17T04:27:57.818 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:27:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:27:57.557+0000 7f7827330700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00880427 s, 1.2 GB/s 2024-05-17T04:27:58.225 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-17T04:27:58.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:58.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:27:58.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T04:27:58.229 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.229 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.229 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:58 smithi110 ceph-mon[35497]: from='osd.5 ' entity='osd.5' 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:58 smithi105 ceph-mon[34795]: from='osd.5 ' entity='osd.5' 2024-05-17T04:27:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:27:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:27:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: osdmap e35: 6 total, 5 up, 6 in 2024-05-17T04:27:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:58.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:58 smithi050 ceph-mon[31739]: from='osd.5 ' entity='osd.5' 2024-05-17T04:27:59.069 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] boot 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:59.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:27:59 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:27:59.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:27:59.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] boot 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:59.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:27:59 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.319 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:27:59.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:27:59.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: osd.5 [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] boot 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: osdmap e36: 6 total, 6 up, 6 in 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: osdmap e37: 6 total, 6 up, 6 in 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:27:59.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:27:59 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:00.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:00 smithi105 ceph-mon[34795]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:28:00.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:00 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi110 2024-05-17T04:28:00.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:00 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-17T04:28:00.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:00 smithi105 ceph-mon[34795]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T04:28:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:00 smithi110 ceph-mon[35497]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:28:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:00 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi110 2024-05-17T04:28:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:00 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-17T04:28:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:00 smithi110 ceph-mon[35497]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T04:28:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:00 smithi050 ceph-mon[31739]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-17T04:28:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:00 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi110 2024-05-17T04:28:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:00 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-17T04:28:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:00 smithi050 ceph-mon[31739]: osdmap e38: 6 total, 6 up, 6 in 2024-05-17T04:28:01.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1153507757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc102b90 msgr2=0x7fa0dc102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:01.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1153507757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc102b90 0x7fa0dc102fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0099c0 tx=0x7fa0cc009cd0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1153507757 shutdown_connections 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1153507757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc104660 0x7fa0dc10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1153507757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc102b90 0x7fa0dc102fa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1153507757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 0x7fa0dc1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1153507757 >> 172.21.15.50:0/1153507757 conn(0x7fa0dc0fe3c0 msgr2=0x7fa0dc100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1153507757 shutdown_connections 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1153507757 wait complete. 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 Processor -- start 2024-05-17T04:28:01.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 -- start start 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 0x7fa0dc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 0x7fa0dc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e510 con 0x7fa0dc104660 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e390 con 0x7fa0dc102b90 2024-05-17T04:28:01.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e23a6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10e690 con 0x7fa0dc103cc0 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e0943700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0dbfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 0x7fa0dc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e0943700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36044/0 (socket says 172.21.15.50:36044) 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.090+0000 7fa0e0943700 1 -- 172.21.15.50:0/1298000299 learned_addr learned my addr 172.21.15.50:0/1298000299 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0db7fe700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 0x7fa0dc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 -- 172.21.15.50:0/1298000299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 msgr2=0x7fa0dc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 0x7fa0dc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 -- 172.21.15.50:0/1298000299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 msgr2=0x7fa0dc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 0x7fa0dc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 -- 172.21.15.50:0/1298000299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1a0c50 con 0x7fa0dc104660 2024-05-17T04:28:01.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0dbfff700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 0x7fa0dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e0943700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ef30 tx=0x7fa0d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40092e0 con 0x7fa0dc104660 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0d4004bb0 con 0x7fa0dc104660 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1a0ee0 con 0x7fa0dc104660 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4004020 con 0x7fa0dc104660 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.091+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1a1340 con 0x7fa0dc104660 2024-05-17T04:28:01.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.092+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc102fa0 con 0x7fa0dc104660 2024-05-17T04:28:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.094+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa0d401b030 con 0x7fa0dc104660 2024-05-17T04:28:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.095+0000 7fa0d97fa700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 0x7fa0c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.095+0000 7fa0dbfff700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 0x7fa0c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.095+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fa0d40e1940 con 0x7fa0dc104660 2024-05-17T04:28:01.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.096+0000 7fa0dbfff700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 0x7fa0c40b4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009990 tx=0x7fa0cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:01.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.099+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0d4022020 con 0x7fa0dc104660 2024-05-17T04:28:01.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:01.323+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa0dc064810 con 0x7fa0c40b2880 2024-05-17T04:28:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:02 smithi105 ceph-mon[34795]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:02 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:02 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:02 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:02 smithi110 ceph-mon[35497]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:02 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:02 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:02 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:02 smithi050 ceph-mon[31739]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:02 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:02 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:02 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:03.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:03 smithi105 ceph-mon[34795]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:03 smithi110 ceph-mon[35497]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:03.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:03 smithi050 ceph-mon[31739]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:04 smithi105 ceph-mon[34795]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:04.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:04 smithi110 ceph-mon[35497]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:04 smithi050 ceph-mon[31739]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:05.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:05 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]: dispatch 2024-05-17T04:28:05.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:05 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]': finished 2024-05-17T04:28:05.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:05 smithi105 ceph-mon[34795]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T04:28:05.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:05.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:05 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3773290657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:05 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]: dispatch 2024-05-17T04:28:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:05 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]': finished 2024-05-17T04:28:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:05 smithi110 ceph-mon[35497]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T04:28:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:05 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3773290657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:05.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:05 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]: dispatch 2024-05-17T04:28:05.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:05 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1629022312' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78770dde-1e76-4d87-b515-1127983aa2c0"}]': finished 2024-05-17T04:28:05.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:05 smithi050 ceph-mon[31739]: osdmap e39: 7 total, 6 up, 7 in 2024-05-17T04:28:05.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:05.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:05 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3773290657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:06.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:06 smithi105 ceph-mon[34795]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:28:06.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:06 smithi110 ceph-mon[35497]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:28:06.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:06 smithi050 ceph-mon[31739]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:28:08.315 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:08 smithi050 ceph-mon[31739]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-17T04:28:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:08 smithi110 ceph-mon[35497]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-17T04:28:08.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:08 smithi105 ceph-mon[34795]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-17T04:28:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:10 smithi050 ceph-mon[31739]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-17T04:28:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:10 smithi105 ceph-mon[34795]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-17T04:28:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:10 smithi110 ceph-mon[35497]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-17T04:28:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:12 smithi110 ceph-mon[35497]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T04:28:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:12 smithi050 ceph-mon[31739]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T04:28:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:12 smithi105 ceph-mon[34795]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-17T04:28:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:13.562 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:13 smithi050 ceph-mon[31739]: Deploying daemon osd.6 on smithi050 2024-05-17T04:28:13.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:13 smithi105 ceph-mon[34795]: Deploying daemon osd.6 on smithi050 2024-05-17T04:28:13.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:13 smithi110 ceph-mon[35497]: Deploying daemon osd.6 on smithi050 2024-05-17T04:28:14.236 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:14 smithi050 ceph-mon[31739]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:14.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:14 smithi105 ceph-mon[34795]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:14 smithi110 ceph-mon[35497]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:28:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:16 smithi050 ceph-mon[31739]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:16.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:16 smithi105 ceph-mon[34795]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:16.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:16 smithi110 ceph-mon[35497]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:18 smithi050 ceph-mon[31739]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:18 smithi105 ceph-mon[34795]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:18.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:18 smithi110 ceph-mon[35497]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:19.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:19.430 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:19 smithi050 ceph-mon[31739]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:19.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:19 smithi105 ceph-mon[34795]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:19.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:19.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:19.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:19 smithi110 ceph-mon[35497]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:22.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.086+0000 7fa0d97fa700 1 -- 172.21.15.50:0/1298000299 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa0dc064810 con 0x7fa0c40b2880 2024-05-17T04:28:22.087 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 6 on host 'smithi050' 2024-05-17T04:28:22.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 msgr2=0x7fa0c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:22.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 0x7fa0c40b4d90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009990 tx=0x7fa0cc01a040 comp rx=0 tx=0).stop 2024-05-17T04:28:22.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 msgr2=0x7fa0dc1a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ef30 tx=0x7fa0d400c3a0 comp rx=0 tx=0).stop 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 shutdown_connections 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa0c40b2880 0x7fa0c40b4d90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0dc104660 0x7fa0dc1a05d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0dc102b90 0x7fa0dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 --2- 172.21.15.50:0/1298000299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0dc103cc0 0x7fa0dc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 >> 172.21.15.50:0/1298000299 conn(0x7fa0dc0fe3c0 msgr2=0x7fa0dc106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 shutdown_connections 2024-05-17T04:28:22.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:28:22.089+0000 7fa0e23a6700 1 -- 172.21.15.50:0/1298000299 wait complete. 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:22.735 DEBUG:teuthology.orchestra.run.smithi050:osd.6> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.6.service 2024-05-17T04:28:22.738 INFO:tasks.cephadm:Deploying osd.7 on smithi105 with /dev/vg_nvme/lv_2... 2024-05-17T04:28:22.738 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T04:28:22.987 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:28:23.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:23 smithi105 ceph-mon[34795]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:23 smithi105 ceph-mon[34795]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:23 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:23.259 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:23 smithi050 ceph-mon[31739]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.259 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:23 smithi050 ceph-mon[31739]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.259 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.259 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.259 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:23 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:23 smithi110 ceph-mon[35497]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:23 smithi110 ceph-mon[35497]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-17T04:28:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:23.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:23 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.135 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:24 smithi050 ceph-mon[31739]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.135 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:28:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:28:24.025+0000 7f2c55feb700 -1 osd.6 0 waiting for initial osdmap 2024-05-17T04:28:24.136 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:28:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:28:24.033+0000 7f2c51da0700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.276 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:24 smithi105 ceph-mon[34795]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: osdmap e40: 7 total, 6 up, 7 in 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: from='osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:24 smithi110 ceph-mon[35497]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:28:24.984 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T04:28:24.984 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T04:28:24.984 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-05-17T04:28:24.984 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-05-17T04:28:24.985 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123615 s, 848 MB/s 2024-05-17T04:28:24.985 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-05-17T04:28:25.285 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:25 smithi105 ceph-mon[34795]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:28:25.285 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:25 smithi105 ceph-mon[34795]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T04:28:25.285 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.285 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:25 smithi050 ceph-mon[31739]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:28:25.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:25 smithi050 ceph-mon[31739]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T04:28:25.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.313 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:25 smithi110 ceph-mon[35497]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:28:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:25 smithi110 ceph-mon[35497]: osdmap e41: 7 total, 6 up, 7 in 2024-05-17T04:28:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:25.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-05-17T04:28:25.885 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:28:26.263 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] boot 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.264 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:26.265 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:26.265 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:28:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:28:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] boot 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:26.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:28:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:28:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-17T04:28:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: osd.6 [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] boot 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: osdmap e42: 7 total, 7 up, 7 in 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:26.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:27 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi050 2024-05-17T04:28:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:27 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-17T04:28:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:27 smithi105 ceph-mon[34795]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T04:28:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:27 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi050 2024-05-17T04:28:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:27 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-17T04:28:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:27 smithi110 ceph-mon[35497]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T04:28:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:27 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi050 2024-05-17T04:28:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:27 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-17T04:28:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:27 smithi050 ceph-mon[31739]: osdmap e43: 7 total, 7 up, 7 in 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 -- 172.21.15.105:0/748231563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c103dd0 msgr2=0x7fb43c10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 --2- 172.21.15.105:0/748231563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c103dd0 0x7fb43c10a680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb43400c210 tx=0x7fb43400c520 comp rx=0 tx=0).stop 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 -- 172.21.15.105:0/748231563 shutdown_connections 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 --2- 172.21.15.105:0/748231563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c103dd0 0x7fb43c10a680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 --2- 172.21.15.105:0/748231563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c102300 0x7fb43c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 --2- 172.21.15.105:0/748231563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 0x7fb43c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 -- 172.21.15.105:0/748231563 >> 172.21.15.105:0/748231563 conn(0x7fb43c0fdb50 msgr2=0x7fb43c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:28:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 -- 172.21.15.105:0/748231563 shutdown_connections 2024-05-17T04:28:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.571+0000 7fb44228a700 1 -- 172.21.15.105:0/748231563 wait complete. 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 Processor -- start 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 -- start start 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 0x7fb43c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c103dd0 0x7fb43c1a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d410 con 0x7fb43c103dd0 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d290 con 0x7fb43c102300 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb44228a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c10d590 con 0x7fb43c103430 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb43bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb43bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48984/0 (socket says 172.21.15.105:48984) 2024-05-17T04:28:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb43bfff700 1 -- 172.21.15.105:0/3706337013 learned_addr learned my addr 172.21.15.105:0/3706337013 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.572+0000 7fb440827700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c103dd0 0x7fb43c1a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb433fff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 0x7fb43c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 -- 172.21.15.105:0/3706337013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 msgr2=0x7fb43c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 0x7fb43c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 -- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c103dd0 msgr2=0x7fb43c1a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c103dd0 0x7fb43c1a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 -- 172.21.15.105:0/3706337013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c1a3f30 con 0x7fb43c102300 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb43bfff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d980 tx=0x7fb42c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c017070 con 0x7fb43c102300 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb42c009dc0 con 0x7fb43c102300 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c1a4220 con 0x7fb43c102300 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00f3c0 con 0x7fb43c102300 2024-05-17T04:28:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.573+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a47e0 con 0x7fb43c102300 2024-05-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.574+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c102710 con 0x7fb43c102300 2024-05-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.576+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb42c009930 con 0x7fb43c102300 2024-05-17T04:28:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.577+0000 7fb439ffb700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 0x7fb4280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.577+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fb42c0e0fe0 con 0x7fb43c102300 2024-05-17T04:28:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.577+0000 7fb433fff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 0x7fb4280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.578+0000 7fb433fff700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 0x7fb4280b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.580+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb42c0a9a10 con 0x7fb43c102300 2024-05-17T04:28:27.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:27.793+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb43c064720 con 0x7fb4280b2880 2024-05-17T04:28:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:28 smithi105 ceph-mon[34795]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:28 smithi110 ceph-mon[35497]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:28 smithi050 ceph-mon[31739]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:29.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:29 smithi105 ceph-mon[34795]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:29.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:29 smithi110 ceph-mon[35497]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:29.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:29 smithi050 ceph-mon[31739]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:30.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:30 smithi105 ceph-mon[34795]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:30.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:30 smithi110 ceph-mon[35497]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:30 smithi050 ceph-mon[31739]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:31 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/4154384261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:31 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:31 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]': finished 2024-05-17T04:28:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:31 smithi105 ceph-mon[34795]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T04:28:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:31 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/4154384261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:31 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:31 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]': finished 2024-05-17T04:28:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:31 smithi110 ceph-mon[35497]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T04:28:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:31 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/4154384261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:31 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]: dispatch 2024-05-17T04:28:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:31 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d035f38-f555-490f-b5bf-018aef1dbb18"}]': finished 2024-05-17T04:28:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:31 smithi050 ceph-mon[31739]: osdmap e44: 8 total, 7 up, 8 in 2024-05-17T04:28:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:32 smithi105 ceph-mon[34795]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:32 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/3431098063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:32 smithi110 ceph-mon[35497]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:32 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/3431098063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:32 smithi050 ceph-mon[31739]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:32 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/3431098063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:34 smithi105 ceph-mon[34795]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:34 smithi110 ceph-mon[35497]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:34 smithi050 ceph-mon[31739]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:36.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:36 smithi105 ceph-mon[34795]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:36.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:36 smithi110 ceph-mon[35497]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:36.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:36 smithi050 ceph-mon[31739]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:38.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:38 smithi105 ceph-mon[34795]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:38.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T04:28:38.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:38.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:38 smithi110 ceph-mon[35497]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:38.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T04:28:38.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:38.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:38 smithi050 ceph-mon[31739]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:38.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-17T04:28:38.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:39.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:39 smithi105 ceph-mon[34795]: Deploying daemon osd.7 on smithi105 2024-05-17T04:28:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:39 smithi110 ceph-mon[35497]: Deploying daemon osd.7 on smithi105 2024-05-17T04:28:39.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:39 smithi050 ceph-mon[31739]: Deploying daemon osd.7 on smithi105 2024-05-17T04:28:40.248 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:40 smithi105 ceph-mon[34795]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:40 smithi110 ceph-mon[35497]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:40 smithi050 ceph-mon[31739]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:42.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:42 smithi105 ceph-mon[34795]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:42 smithi110 ceph-mon[35497]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:42.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:42 smithi050 ceph-mon[31739]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:44 smithi105 ceph-mon[34795]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:44 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:44 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:44 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:44 smithi110 ceph-mon[35497]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:44 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:44 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:44 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:44.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:44 smithi050 ceph-mon[31739]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:44.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:44 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:44.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:44 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:44.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:44 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:46.230 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:46 smithi105 ceph-mon[34795]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:46.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:46 smithi050 ceph-mon[31739]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:46 smithi110 ceph-mon[35497]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:47.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.915+0000 7fb439ffb700 1 -- 172.21.15.105:0/3706337013 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb43c064720 con 0x7fb4280b2880 2024-05-17T04:28:47.916 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 7 on host 'smithi105' 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 msgr2=0x7fb4280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 0x7fb4280b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb424006fd0 tx=0x7fb424008040 comp rx=0 tx=0).stop 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 msgr2=0x7fb43c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d980 tx=0x7fb42c00dc90 comp rx=0 tx=0).stop 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 shutdown_connections 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4280b2880 0x7fb4280b4d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb43c103dd0 0x7fb43c1a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb43c102300 0x7fb43c19c3d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:47.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 --2- 172.21.15.105:0/3706337013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb43c103430 0x7fb43c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 >> 172.21.15.105:0/3706337013 conn(0x7fb43c0fdb50 msgr2=0x7fb43c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:28:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 shutdown_connections 2024-05-17T04:28:47.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:28:47.917+0000 7fb44228a700 1 -- 172.21.15.105:0/3706337013 wait complete. 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:48.301 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.302 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.302 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.302 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.302 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.302 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-17T04:28:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:48.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:28:48.618 DEBUG:teuthology.orchestra.run.smithi105:osd.7> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.7.service 2024-05-17T04:28:48.620 INFO:tasks.cephadm:Deploying osd.8 on smithi110 with /dev/vg_nvme/lv_2... 2024-05-17T04:28:48.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-17T04:28:48.866 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:28:49.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:49 smithi105 ceph-mon[34795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T04:28:49.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:49 smithi105 ceph-mon[34795]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T04:28:49.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:49.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:49 smithi105 ceph-mon[34795]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:49.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:49 smithi105 ceph-mon[34795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:49.180 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:28:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:28:49.081+0000 7f96bd47b700 -1 osd.7 0 waiting for initial osdmap 2024-05-17T04:28:49.180 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:28:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:28:49.091+0000 7f96b9230700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:28:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:49 smithi110 ceph-mon[35497]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T04:28:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:49 smithi110 ceph-mon[35497]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T04:28:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:49 smithi110 ceph-mon[35497]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:49.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:49 smithi110 ceph-mon[35497]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:49.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:49 smithi050 ceph-mon[31739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-17T04:28:49.418 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:49 smithi050 ceph-mon[31739]: osdmap e45: 8 total, 7 up, 8 in 2024-05-17T04:28:49.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:49.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:49 smithi050 ceph-mon[31739]: from='osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:49.419 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:49 smithi050 ceph-mon[31739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:28:50.155 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:50 smithi105 ceph-mon[34795]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:50.155 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:50 smithi105 ceph-mon[34795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:28:50.155 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:50 smithi105 ceph-mon[34795]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T04:28:50.155 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.155 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:50 smithi110 ceph-mon[35497]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:50 smithi110 ceph-mon[35497]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:28:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:50 smithi110 ceph-mon[35497]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T04:28:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.403 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:50 smithi050 ceph-mon[31739]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-17T04:28:50.403 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:50 smithi050 ceph-mon[31739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:28:50.403 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:50 smithi050 ceph-mon[31739]: osdmap e46: 8 total, 7 up, 8 in 2024-05-17T04:28:50.403 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.403 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00836159 s, 1.3 GB/s 2024-05-17T04:28:50.916 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-17T04:28:51.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:51 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:28:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:51 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:28:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:51 smithi105 ceph-mon[34795]: osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] boot 2024-05-17T04:28:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:51 smithi105 ceph-mon[34795]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T04:28:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:51.502 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:51 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:28:51.503 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:51 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:28:51.503 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:51 smithi110 ceph-mon[35497]: osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] boot 2024-05-17T04:28:51.503 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:51 smithi110 ceph-mon[35497]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T04:28:51.503 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:51.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-05-17T04:28:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:51 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:28:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:51 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:28:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:51 smithi050 ceph-mon[31739]: osd.7 [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] boot 2024-05-17T04:28:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:51 smithi050 ceph-mon[31739]: osdmap e47: 8 total, 8 up, 8 in 2024-05-17T04:28:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-17T04:28:51.767 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:28:52.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:52.206 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:52.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:52.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: osdmap e48: 8 total, 8 up, 8 in 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:28:52.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:52.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:52.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:28:52.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:28:53.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 -- 172.21.15.110:0/3115163439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4103cc0 msgr2=0x7fc6b41040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:53.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/3115163439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4103cc0 0x7fc6b41040d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80099c0 tx=0x7fc6a8009cd0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 -- 172.21.15.110:0/3115163439 shutdown_connections 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/3115163439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4104660 0x7fc6b410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/3115163439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4102b90 0x7fc6b4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/3115163439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4103cc0 0x7fc6b41040d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 -- 172.21.15.110:0/3115163439 >> 172.21.15.110:0/3115163439 conn(0x7fc6b40fe3c0 msgr2=0x7fc6b4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 -- 172.21.15.110:0/3115163439 shutdown_connections 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.311+0000 7fc6ba27f700 1 -- 172.21.15.110:0/3115163439 wait complete. 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 Processor -- start 2024-05-17T04:28:53.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 -- start start 2024-05-17T04:28:53.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4102b90 0x7fc6b419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:53.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 0x7fc6b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:53.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:53.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410e4b0 con 0x7fc6b4103cc0 2024-05-17T04:28:53.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410e330 con 0x7fc6b4102b90 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6ba27f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410e630 con 0x7fc6b4104660 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6b881c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6b881c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33194/0 (socket says 172.21.15.110:33194) 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6b881c700 1 -- 172.21.15.110:0/1512095196 learned_addr learned my addr 172.21.15.110:0/1512095196 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6b37fe700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 0x7fc6b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.312+0000 7fc6b3fff700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4102b90 0x7fc6b419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 -- 172.21.15.110:0/1512095196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4102b90 msgr2=0x7fc6b419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4102b90 0x7fc6b419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 -- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 msgr2=0x7fc6b419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 0x7fc6b419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 -- 172.21.15.110:0/1512095196 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b41a1d70 con 0x7fc6b4104660 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b37fe700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 0x7fc6b419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:28:53.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b881c700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00ada0 tx=0x7fc6ac00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac017020 con 0x7fc6b4104660 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6ac00cd60 con 0x7fc6b4104660 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b41a2060 con 0x7fc6b4104660 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac005280 con 0x7fc6b4104660 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.313+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b41a2550 con 0x7fc6b4104660 2024-05-17T04:28:53.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.314+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b4102fa0 con 0x7fc6b4104660 2024-05-17T04:28:53.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.316+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc6ac00c8d0 con 0x7fc6b4104660 2024-05-17T04:28:53.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.317+0000 7fc6b17fa700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 0x7fc69c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:28:53.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.317+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7fc6ac0e1430 con 0x7fc6b4104660 2024-05-17T04:28:53.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.317+0000 7fc6b3fff700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 0x7fc69c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:28:53.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.318+0000 7fc6b3fff700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 0x7fc69c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005950 tx=0x7fc6a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:28:53.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.320+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6ac021020 con 0x7fc6b4104660 2024-05-17T04:28:53.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:28:53.531+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc6b4100570 con 0x7fc69c0b2880 2024-05-17T04:28:53.532 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:53 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi105 2024-05-17T04:28:53.532 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:53 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 5696M 2024-05-17T04:28:53.533 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:53 smithi110 ceph-mon[35497]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T04:28:53.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:53 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi105 2024-05-17T04:28:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:53 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 5696M 2024-05-17T04:28:53.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:53 smithi105 ceph-mon[34795]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T04:28:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:53 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi105 2024-05-17T04:28:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:53 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 5696M 2024-05-17T04:28:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:53 smithi050 ceph-mon[31739]: osdmap e49: 8 total, 8 up, 8 in 2024-05-17T04:28:54.425 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:54 smithi110 ceph-mon[35497]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:54.425 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:54 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:54.425 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:54 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:54.425 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:54 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:54 smithi105 ceph-mon[34795]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:54.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:54 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:54.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:54 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:54.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:54 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:54 smithi050 ceph-mon[31739]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:54 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:28:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:54 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:28:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:54 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:28:55.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:55 smithi110 ceph-mon[35497]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:55.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:55 smithi105 ceph-mon[34795]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:55.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:55 smithi050 ceph-mon[31739]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:28:56.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:56 smithi105 ceph-mon[34795]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:56 smithi110 ceph-mon[35497]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:56.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:56 smithi050 ceph-mon[31739]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/4172407976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]': finished 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:28:57.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:57 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/2505233107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/4172407976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]': finished 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:28:57.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:57 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/2505233107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/4172407976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]: dispatch 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0df6cef8-8d82-478a-98a1-980848605ac9"}]': finished 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: osdmap e50: 9 total, 8 up, 9 in 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:28:57.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:57 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/2505233107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:28:58.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:28:58 smithi105 ceph-mon[34795]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:28:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:28:58 smithi110 ceph-mon[35497]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:28:58.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:28:58 smithi050 ceph-mon[31739]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:29:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:00 smithi110 ceph-mon[35497]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-17T04:29:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:00 smithi050 ceph-mon[31739]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-17T04:29:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:00 smithi105 ceph-mon[34795]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-17T04:29:02.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:02 smithi110 ceph-mon[35497]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-17T04:29:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:02 smithi105 ceph-mon[34795]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-17T04:29:02.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:02 smithi050 ceph-mon[31739]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-17T04:29:04.380 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:04 smithi110 ceph-mon[35497]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:04.380 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T04:29:04.380 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:04.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:04 smithi105 ceph-mon[34795]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:04.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T04:29:04.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:04 smithi050 ceph-mon[31739]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-17T04:29:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:05.384 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:05 smithi110 ceph-mon[35497]: Deploying daemon osd.8 on smithi110 2024-05-17T04:29:05.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:05 smithi105 ceph-mon[34795]: Deploying daemon osd.8 on smithi110 2024-05-17T04:29:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:05 smithi050 ceph-mon[31739]: Deploying daemon osd.8 on smithi110 2024-05-17T04:29:06.425 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:06 smithi110 ceph-mon[35497]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:06 smithi105 ceph-mon[34795]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:06.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:06 smithi050 ceph-mon[31739]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:07.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:07 smithi110 ceph-mon[35497]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:07.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:07 smithi105 ceph-mon[34795]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:07.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:07 smithi050 ceph-mon[31739]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:10.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:09 smithi105 ceph-mon[34795]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:10.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:09 smithi110 ceph-mon[35497]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:09 smithi050 ceph-mon[31739]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:11.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:11 smithi110 ceph-mon[35497]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:12.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:11 smithi105 ceph-mon[34795]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:12.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:11 smithi050 ceph-mon[31739]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:13.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.006+0000 7fc6b17fa700 1 -- 172.21.15.110:0/1512095196 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6b4100570 con 0x7fc69c0b2880 2024-05-17T04:29:13.006 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 8 on host 'smithi110' 2024-05-17T04:29:13.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.008+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 msgr2=0x7fc69c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:13.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.008+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 0x7fc69c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005950 tx=0x7fc6a400b410 comp rx=0 tx=0).stop 2024-05-17T04:29:13.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.008+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 msgr2=0x7fc6b41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:13.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.008+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00ada0 tx=0x7fc6ac00c330 comp rx=0 tx=0).stop 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 shutdown_connections 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6b4104660 0x7fc6b41a16f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6b4102b90 0x7fc6b419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc69c0b2880 0x7fc69c0b4d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 --2- 172.21.15.110:0/1512095196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b4103cc0 0x7fc6b419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 >> 172.21.15.110:0/1512095196 conn(0x7fc6b40fe3c0 msgr2=0x7fc6b40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 shutdown_connections 2024-05-17T04:29:13.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:29:13.009+0000 7fc6ba27f700 1 -- 172.21.15.110:0/1512095196 wait complete. 2024-05-17T04:29:13.709 DEBUG:teuthology.orchestra.run.smithi110:osd.8> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.8.service 2024-05-17T04:29:13.711 INFO:tasks.cephadm:Deploying osd.9 on smithi050 with /dev/vg_nvme/lv_1... 2024-05-17T04:29:13.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T04:29:13.973 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:14.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:13 smithi050 ceph-mon[31739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:14.268 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:13 smithi110 ceph-mon[35497]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:13 smithi105 ceph-mon[34795]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:15 smithi050 ceph-mon[31739]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T04:29:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T04:29:15.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:15.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:15 smithi110 ceph-mon[35497]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: osdmap e51: 9 total, 8 up, 9 in 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: from='osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:29:15.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:15 smithi105 ceph-mon[34795]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:16.191 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T04:29:16.191 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T04:29:16.192 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-17T04:29:16.192 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-17T04:29:16.192 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849751 s, 1.2 GB/s 2024-05-17T04:29:16.192 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-17T04:29:16.318 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:29:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:29:15.861+0000 7fbcf7c32700 -1 osd.8 0 waiting for initial osdmap 2024-05-17T04:29:16.318 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:29:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:29:15.872+0000 7fbcef9df700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:29:16.514 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:16 smithi050 ceph-mon[31739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:29:16.515 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:16 smithi050 ceph-mon[31739]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T04:29:16.515 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.515 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.515 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:16 smithi050 ceph-mon[31739]: from='osd.8 ' entity='osd.8' 2024-05-17T04:29:16.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:16 smithi105 ceph-mon[34795]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:29:16.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:16 smithi105 ceph-mon[34795]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T04:29:16.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:16 smithi105 ceph-mon[34795]: from='osd.8 ' entity='osd.8' 2024-05-17T04:29:16.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:16 smithi110 ceph-mon[35497]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:29:16.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:16 smithi110 ceph-mon[35497]: osdmap e52: 9 total, 8 up, 9 in 2024-05-17T04:29:16.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:16.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:16 smithi110 ceph-mon[35497]: from='osd.8 ' entity='osd.8' 2024-05-17T04:29:16.980 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-05-17T04:29:17.237 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] boot 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi110 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.628 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.629 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:17 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:29:17.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:29:17.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] boot 2024-05-17T04:29:17.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T04:29:17.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi110 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:17.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:17 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: osd.8 [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] boot 2024-05-17T04:29:17.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: osdmap e53: 9 total, 9 up, 9 in 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi110 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:17.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:17.819 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:17.819 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:17.819 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:17 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:18.697 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:18 smithi050 ceph-mon[31739]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T04:29:18.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:18 smithi105 ceph-mon[34795]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T04:29:18.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:18 smithi110 ceph-mon[35497]: osdmap e54: 9 total, 9 up, 9 in 2024-05-17T04:29:19.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 -- 172.21.15.50:0/2462533344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0104e60 msgr2=0x7f7eb0105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:19.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 --2- 172.21.15.50:0/2462533344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0104e60 0x7f7eb0105270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40099c0 tx=0x7f7ea4009cd0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 -- 172.21.15.50:0/2462533344 shutdown_connections 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 --2- 172.21.15.50:0/2462533344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 0x7f7eb010c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 --2- 172.21.15.50:0/2462533344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0103d30 0x7f7eb0104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 --2- 172.21.15.50:0/2462533344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0104e60 0x7f7eb0105270 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 -- 172.21.15.50:0/2462533344 >> 172.21.15.50:0/2462533344 conn(0x7f7eb00ff560 msgr2=0x7f7eb0101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 -- 172.21.15.50:0/2462533344 shutdown_connections 2024-05-17T04:29:19.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.096+0000 7f7eb513f700 1 -- 172.21.15.50:0/2462533344 wait complete. 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 Processor -- start 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 -- start start 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0104e60 0x7f7eb019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 0x7f7eb01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010f650 con 0x7f7eb0103d30 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010f4d0 con 0x7f7eb0105800 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eb513f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb010f7d0 con 0x7f7eb0104e60 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eaed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eaed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39336/0 (socket says 172.21.15.50:39336) 2024-05-17T04:29:19.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.097+0000 7f7eaed9d700 1 -- 172.21.15.50:0/1020168049 learned_addr learned my addr 172.21.15.50:0/1020168049 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaf59e700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 0x7f7eb01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eae59c700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0104e60 0x7f7eb019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 -- 172.21.15.50:0/1020168049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0104e60 msgr2=0x7f7eb019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0104e60 0x7f7eb019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 -- 172.21.15.50:0/1020168049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 msgr2=0x7f7eb01a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 0x7f7eb01a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 -- 172.21.15.50:0/1020168049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01a1d90 con 0x7f7eb0103d30 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eaed9d700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000d980 tx=0x7f7ea000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0017070 con 0x7f7eb0103d30 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ea0009dc0 con 0x7f7eb0103d30 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb0075ac0 con 0x7f7eb0103d30 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000f3c0 con 0x7f7eb0103d30 2024-05-17T04:29:19.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.098+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb0076000 con 0x7f7eb0103d30 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.099+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb0104140 con 0x7f7eb0103d30 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.101+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7ea0009930 con 0x7f7eb0103d30 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.102+0000 7f7e97fff700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 0x7f7e980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.102+0000 7f7eae59c700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 0x7f7e980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.103+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f7ea00e1400 con 0x7f7eb0103d30 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.103+0000 7f7eae59c700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 0x7f7e980b4d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7eb019b2e0 tx=0x7f7ea4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:29:19.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.106+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ea00a9a30 con 0x7f7eb0103d30 2024-05-17T04:29:19.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:19.332+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7eb00777a0 con 0x7f7e980b2880 2024-05-17T04:29:19.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:19 smithi105 ceph-mon[34795]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T04:29:19.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:19 smithi105 ceph-mon[34795]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:19.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:19.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:19.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:19.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:19 smithi110 ceph-mon[35497]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T04:29:19.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:19 smithi110 ceph-mon[35497]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:19.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:19.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:19.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:19.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:19 smithi050 ceph-mon[31739]: osdmap e55: 9 total, 9 up, 9 in 2024-05-17T04:29:19.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:19 smithi050 ceph-mon[31739]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:19.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:19.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:19.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:20.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:20 smithi050 ceph-mon[31739]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:20.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:20 smithi105 ceph-mon[34795]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:20.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:20 smithi110 ceph-mon[35497]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:21.535 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:21 smithi050 ceph-mon[31739]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-17T04:29:21.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:21 smithi105 ceph-mon[34795]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-17T04:29:21.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:21 smithi110 ceph-mon[35497]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-17T04:29:22.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:22 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]: dispatch 2024-05-17T04:29:22.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:22 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]': finished 2024-05-17T04:29:22.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:22 smithi050 ceph-mon[31739]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T04:29:22.710 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:22.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:22 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]: dispatch 2024-05-17T04:29:22.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:22 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]': finished 2024-05-17T04:29:22.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:22 smithi105 ceph-mon[34795]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T04:29:22.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:22.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:22 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]: dispatch 2024-05-17T04:29:22.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:22 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/950231283' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23c1e66f-5c60-40f9-a478-e9a4e43517ac"}]': finished 2024-05-17T04:29:22.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:22 smithi110 ceph-mon[35497]: osdmap e56: 10 total, 9 up, 10 in 2024-05-17T04:29:22.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:23.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:23 smithi105 ceph-mon[34795]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:29:23.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:23 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/154370783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:23.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:23 smithi110 ceph-mon[35497]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:29:23.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:23 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/154370783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:23.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:23 smithi050 ceph-mon[31739]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-17T04:29:23.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:23 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/154370783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:26.064 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:25 smithi050 ceph-mon[31739]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-17T04:29:26.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:25 smithi105 ceph-mon[34795]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-17T04:29:26.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:25 smithi110 ceph-mon[35497]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-17T04:29:27.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:27 smithi050 ceph-mon[31739]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-17T04:29:28.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:27 smithi105 ceph-mon[34795]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-17T04:29:28.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:27 smithi110 ceph-mon[35497]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-17T04:29:30.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:29 smithi105 ceph-mon[34795]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:29 smithi110 ceph-mon[35497]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:29 smithi050 ceph-mon[31739]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:29:31.014 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T04:29:31.014 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T04:29:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:31.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-17T04:29:31.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:31 smithi105 ceph-mon[34795]: Deploying daemon osd.9 on smithi050 2024-05-17T04:29:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:31 smithi105 ceph-mon[34795]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:31 smithi110 ceph-mon[35497]: Deploying daemon osd.9 on smithi050 2024-05-17T04:29:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:31 smithi110 ceph-mon[35497]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:32.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:31 smithi050 ceph-mon[31739]: Deploying daemon osd.9 on smithi050 2024-05-17T04:29:32.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:31 smithi050 ceph-mon[31739]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:34.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:33 smithi105 ceph-mon[34795]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:33 smithi110 ceph-mon[35497]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:33 smithi050 ceph-mon[31739]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:36.022 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:35 smithi050 ceph-mon[31739]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:36.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:35 smithi105 ceph-mon[34795]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:36.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:35 smithi110 ceph-mon[35497]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:37.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:37.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:37 smithi105 ceph-mon[34795]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:37.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:37.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:37 smithi110 ceph-mon[35497]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:37.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:37 smithi050 ceph-mon[31739]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:40.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:40.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:40.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:40.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:40.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:40.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:40.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:40.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:40.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:40.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:40.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.644+0000 7f7e97fff700 1 -- 172.21.15.50:0/1020168049 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7eb00777a0 con 0x7f7e980b2880 2024-05-17T04:29:40.646 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 9 on host 'smithi050' 2024-05-17T04:29:40.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 msgr2=0x7f7e980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:40.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 0x7f7e980b4d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7eb019b2e0 tx=0x7f7ea4005fb0 comp rx=0 tx=0).stop 2024-05-17T04:29:40.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 msgr2=0x7f7eb019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000d980 tx=0x7f7ea000dc90 comp rx=0 tx=0).stop 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 shutdown_connections 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7eb0105800 0x7f7eb01a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7e980b2880 0x7f7e980b4d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7eb0103d30 0x7f7eb019a230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 --2- 172.21.15.50:0/1020168049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0104e60 0x7f7eb019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 >> 172.21.15.50:0/1020168049 conn(0x7f7eb00ff560 msgr2=0x7f7eb0108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 shutdown_connections 2024-05-17T04:29:40.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:29:40.647+0000 7f7eb513f700 1 -- 172.21.15.50:0/1020168049 wait complete. 2024-05-17T04:29:41.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:40 smithi050 ceph-mon[31739]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T04:29:41.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:41.035 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:40 smithi105 ceph-mon[34795]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T04:29:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:41.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:40 smithi110 ceph-mon[35497]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-17T04:29:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:29:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:41.482 DEBUG:teuthology.orchestra.run.smithi050:osd.9> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.9.service 2024-05-17T04:29:41.484 INFO:tasks.cephadm:Deploying osd.10 on smithi105 with /dev/vg_nvme/lv_1... 2024-05-17T04:29:41.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T04:29:41.742 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:29:41.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:41 smithi105 ceph-mon[34795]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:41.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:41 smithi105 ceph-mon[34795]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T04:29:41.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:41 smithi105 ceph-mon[34795]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T04:29:41.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:41 smithi105 ceph-mon[34795]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:29:41.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:42.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:41 smithi050 ceph-mon[31739]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:42.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:41 smithi050 ceph-mon[31739]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T04:29:42.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:41 smithi050 ceph-mon[31739]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T04:29:42.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:41 smithi050 ceph-mon[31739]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:29:42.049 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:41 smithi110 ceph-mon[35497]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:41 smithi110 ceph-mon[35497]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-17T04:29:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:41 smithi110 ceph-mon[35497]: osdmap e57: 10 total, 9 up, 10 in 2024-05-17T04:29:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:41 smithi110 ceph-mon[35497]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-17T04:29:42.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:42.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:29:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:29:42.346+0000 7f08b612c700 -1 osd.9 0 waiting for initial osdmap 2024-05-17T04:29:42.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:29:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:29:42.360+0000 7f08aded9700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:29:43.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:42 smithi050 ceph-mon[31739]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:29:43.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:42 smithi050 ceph-mon[31739]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T04:29:43.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.029 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:42 smithi050 ceph-mon[31739]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' 2024-05-17T04:29:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:42 smithi105 ceph-mon[34795]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:29:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:42 smithi105 ceph-mon[34795]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T04:29:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:42 smithi105 ceph-mon[34795]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' 2024-05-17T04:29:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:42 smithi110 ceph-mon[35497]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-17T04:29:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:42 smithi110 ceph-mon[35497]: osdmap e58: 10 total, 9 up, 10 in 2024-05-17T04:29:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:43.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:42 smithi110 ceph-mon[35497]: from='osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012]' entity='osd.9' 2024-05-17T04:29:43.832 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T04:29:43.832 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T04:29:43.832 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-05-17T04:29:43.832 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-05-17T04:29:43.833 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119473 s, 878 MB/s 2024-05-17T04:29:43.833 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] boot 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T04:29:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] boot 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T04:29:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:29:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:29:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-17T04:29:44.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: osd.9 [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] boot 2024-05-17T04:29:44.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: osdmap e59: 10 total, 10 up, 10 in 2024-05-17T04:29:44.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-17T04:29:44.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-05-17T04:29:44.873 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi050 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-17T04:29:46.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:46.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:46.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T04:29:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi050 2024-05-17T04:29:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:46.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: osdmap e60: 10 total, 10 up, 10 in 2024-05-17T04:29:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi050 2024-05-17T04:29:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:29:46.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:29:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.603+0000 7fe60e0e1700 1 -- 172.21.15.105:0/2468526149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608104660 msgr2=0x7fe60810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.603+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/2468526149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608104660 0x7fe60810af10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe60000b0a0 tx=0x7fe60000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 -- 172.21.15.105:0/2468526149 shutdown_connections 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/2468526149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608104660 0x7fe60810af10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/2468526149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608102b90 0x7fe608102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/2468526149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 0x7fe6081040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 -- 172.21.15.105:0/2468526149 >> 172.21.15.105:0/2468526149 conn(0x7fe6080fe3c0 msgr2=0x7fe608100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 -- 172.21.15.105:0/2468526149 shutdown_connections 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 -- 172.21.15.105:0/2468526149 wait complete. 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 Processor -- start 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.604+0000 7fe60e0e1700 1 -- start start 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 0x7fe60819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608104660 0x7fe6081a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810e580 con 0x7fe608103cc0 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810e400 con 0x7fe608102b90 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810e700 con 0x7fe608104660 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47426/0 (socket says 172.21.15.105:47426) 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 -- 172.21.15.105:0/4021747987 learned_addr learned my addr 172.21.15.105:0/4021747987 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe607fff700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608104660 0x7fe6081a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe606ffd700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 0x7fe60819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 -- 172.21.15.105:0/4021747987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608104660 msgr2=0x7fe6081a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608104660 0x7fe6081a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 -- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 msgr2=0x7fe60819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 0x7fe60819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:29:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 -- 172.21.15.105:0/4021747987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6081a1d50 con 0x7fe608102b90 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe6077fe700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e970 tx=0x7fe5f800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800cbd0 con 0x7fe608102b90 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5f800cd70 con 0x7fe608102b90 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6081a1fe0 con 0x7fe608102b90 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8005570 con 0x7fe608102b90 2024-05-17T04:29:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.605+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a2590 con 0x7fe608102b90 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.606+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe608102fa0 con 0x7fe608102b90 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.608+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe5f8004750 con 0x7fe608102b90 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.608+0000 7fe604ff9700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 0x7fe5f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.609+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fe5f80e1dc0 con 0x7fe608102b90 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.609+0000 7fe606ffd700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 0x7fe5f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:29:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.609+0000 7fe606ffd700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 0x7fe5f40b4e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006fd0 tx=0x7fe5f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:29:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.612+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5f8054df0 con 0x7fe608102b90 2024-05-17T04:29:46.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:29:46.824+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe608100510 con 0x7fe5f40b2900 2024-05-17T04:29:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:47 smithi105 ceph-mon[34795]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:47 smithi105 ceph-mon[34795]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:48.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:47 smithi110 ceph-mon[35497]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:47 smithi110 ceph-mon[35497]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:47 smithi050 ceph-mon[31739]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:47 smithi050 ceph-mon[31739]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:29:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:29:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:29:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/998953810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]': finished 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T04:29:50.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/998953810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]': finished 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T04:29:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:29:50.079 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:50.080 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/998953810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.080 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]: dispatch 2024-05-17T04:29:50.080 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d226622f-07c2-4a51-a54a-74440d67a19d"}]': finished 2024-05-17T04:29:50.080 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: osdmap e61: 11 total, 10 up, 11 in 2024-05-17T04:29:50.080 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:29:51.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:50 smithi105 ceph-mon[34795]: from='client.? 172.21.15.105:0/1954931943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:51.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:50 smithi110 ceph-mon[35497]: from='client.? 172.21.15.105:0/1954931943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:51.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:50 smithi050 ceph-mon[31739]: from='client.? 172.21.15.105:0/1954931943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:29:52.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:51 smithi105 ceph-mon[34795]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:52.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:51 smithi110 ceph-mon[35497]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:51 smithi050 ceph-mon[31739]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:54.051 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:53 smithi105 ceph-mon[34795]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:54.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:53 smithi110 ceph-mon[35497]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:54.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:53 smithi050 ceph-mon[31739]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:55.926 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:55 smithi105 ceph-mon[34795]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:55 smithi110 ceph-mon[35497]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:56.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:55 smithi050 ceph-mon[31739]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:58.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:57 smithi105 ceph-mon[34795]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:58.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T04:29:58.030 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:57 smithi110 ceph-mon[35497]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T04:29:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:57 smithi050 ceph-mon[31739]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:29:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-17T04:29:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:29:59.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:58 smithi105 ceph-mon[34795]: Deploying daemon osd.10 on smithi105 2024-05-17T04:29:59.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:58 smithi110 ceph-mon[35497]: Deploying daemon osd.10 on smithi105 2024-05-17T04:29:59.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:58 smithi050 ceph-mon[31739]: Deploying daemon osd.10 on smithi105 2024-05-17T04:29:59.924 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:29:59 smithi105 ceph-mon[34795]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:00.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:29:59 smithi110 ceph-mon[35497]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:00.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:29:59 smithi050 ceph-mon[31739]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:00.987 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:00 smithi105 ceph-mon[34795]: overall HEALTH_OK 2024-05-17T04:30:01.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:00 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:30:01.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:00 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:30:01.873 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:01 smithi105 ceph-mon[34795]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:02.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:01 smithi110 ceph-mon[35497]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:02.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:01 smithi050 ceph-mon[31739]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:04 smithi105 ceph-mon[34795]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:04 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:04 smithi110 ceph-mon[35497]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:04 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:04 smithi050 ceph-mon[31739]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:04 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:06.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:06 smithi105 ceph-mon[34795]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:06.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:06 smithi110 ceph-mon[35497]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:06.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:06 smithi050 ceph-mon[31739]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:06.644 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 10 on host 'smithi105' 2024-05-17T04:30:06.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.644+0000 7fe604ff9700 1 -- 172.21.15.105:0/4021747987 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe608100510 con 0x7fe5f40b2900 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 msgr2=0x7fe5f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 0x7fe5f40b4e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0006fd0 tx=0x7fe5f0008040 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 msgr2=0x7fe60819a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800e970 tx=0x7fe5f800ec80 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 shutdown_connections 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe608104660 0x7fe6081a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe608102b90 0x7fe60819a1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe5f40b2900 0x7fe5f40b4e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 --2- 172.21.15.105:0/4021747987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe608103cc0 0x7fe60819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 >> 172.21.15.105:0/4021747987 conn(0x7fe6080fe3c0 msgr2=0x7fe6080ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:06.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 shutdown_connections 2024-05-17T04:30:06.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-17T04:30:06.647+0000 7fe60e0e1700 1 -- 172.21.15.105:0/4021747987 wait complete. 2024-05-17T04:30:07.305 DEBUG:teuthology.orchestra.run.smithi105:osd.10> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.10.service 2024-05-17T04:30:07.308 INFO:tasks.cephadm:Deploying osd.11 on smithi110 with /dev/vg_nvme/lv_1... 2024-05-17T04:30:07.308 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-17T04:30:07.568 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:30:07.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:07.917 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:07 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:08.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:08.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:08.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:07 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:08.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:08.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-17T04:30:08.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:07 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:08.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:08 smithi105 ceph-mon[34795]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T04:30:08.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:08 smithi105 ceph-mon[34795]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T04:30:08.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:08.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:08 smithi105 ceph-mon[34795]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:08.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:08 smithi105 ceph-mon[34795]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:08.815 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:30:08 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:30:08.644+0000 7f9513592700 -1 osd.10 0 waiting for initial osdmap 2024-05-17T04:30:08.816 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:30:08 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:30:08.658+0000 7f950f347700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:30:09.004 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:08 smithi110 ceph-mon[35497]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T04:30:09.004 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:08 smithi110 ceph-mon[35497]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T04:30:09.004 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.004 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:08 smithi110 ceph-mon[35497]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:09.004 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:08 smithi110 ceph-mon[35497]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:08 smithi050 ceph-mon[31739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-17T04:30:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:08 smithi050 ceph-mon[31739]: osdmap e62: 11 total, 10 up, 11 in 2024-05-17T04:30:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:08 smithi050 ceph-mon[31739]: from='osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:09.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:08 smithi050 ceph-mon[31739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-05-17T04:30:09.554 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-17T04:30:09.554 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-17T04:30:09.554 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-17T04:30:09.554 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-17T04:30:09.554 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00837674 s, 1.3 GB/s 2024-05-17T04:30:09.555 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] boot 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T04:30:09.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:09 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] boot 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T04:30:09.859 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:09 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-05-17T04:30:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: osdmap e63: 11 total, 10 up, 11 in 2024-05-17T04:30:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:10.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:10.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-17T04:30:10.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: osd.10 [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] boot 2024-05-17T04:30:10.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: osdmap e64: 11 total, 11 up, 11 in 2024-05-17T04:30:10.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:09 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-17T04:30:10.317 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-05-17T04:30:10.558 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:30:11.009 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:10 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:30:11.009 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:10 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:30:11.009 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:10 smithi110 ceph-mon[35497]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T04:30:11.045 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:10 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:30:11.045 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:10 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:30:11.046 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:10 smithi105 ceph-mon[34795]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T04:30:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:10 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:30:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:10 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:30:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:10 smithi050 ceph-mon[31739]: osdmap e65: 11 total, 11 up, 11 in 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi105 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.024 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:12.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:12.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:12.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 -- 172.21.15.110:0/2839778027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0104660 msgr2=0x7f41f010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:12.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 --2- 172.21.15.110:0/2839778027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0104660 0x7f41f010af10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41e800b0d0 tx=0x7f41e800b3e0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 -- 172.21.15.110:0/2839778027 shutdown_connections 2024-05-17T04:30:12.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 --2- 172.21.15.110:0/2839778027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0104660 0x7f41f010af10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 --2- 172.21.15.110:0/2839778027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 0x7f41f0102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 --2- 172.21.15.110:0/2839778027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0103cc0 0x7f41f01040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 -- 172.21.15.110:0/2839778027 >> 172.21.15.110:0/2839778027 conn(0x7f41f00fe3c0 msgr2=0x7f41f0100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 -- 172.21.15.110:0/2839778027 shutdown_connections 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.240+0000 7f41f535c700 1 -- 172.21.15.110:0/2839778027 wait complete. 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 Processor -- start 2024-05-17T04:30:12.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 -- start start 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 0x7f41f0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0104660 0x7f41f01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f41f010e4c0 con 0x7f41f0102b90 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41f010e340 con 0x7f41f0104660 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41f535c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f41f010e640 con 0x7f41f0103cc0 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54938/0 (socket says 172.21.15.110:54938) 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41ee7fc700 1 -- 172.21.15.110:0/3741028664 learned_addr learned my addr 172.21.15.110:0/3741028664 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:30:12.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.241+0000 7f41eeffd700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 0x7f41f0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ef7fe700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0104660 0x7f41f01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 -- 172.21.15.110:0/3741028664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0104660 msgr2=0x7f41f01a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0104660 0x7f41f01a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 -- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 msgr2=0x7f41f0199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 0x7f41f0199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 -- 172.21.15.110:0/3741028664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f006c0f0 con 0x7f41f0103cc0 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41ee7fc700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e400d950 tx=0x7f41e400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4016070 con 0x7f41f0103cc0 2024-05-17T04:30:12.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f006c3e0 con 0x7f41f0103cc0 2024-05-17T04:30:12.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41e4009dd0 con 0x7f41f0103cc0 2024-05-17T04:30:12.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e400f3c0 con 0x7f41f0103cc0 2024-05-17T04:30:12.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.242+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f006c920 con 0x7f41f0103cc0 2024-05-17T04:30:12.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.243+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f0102fa0 con 0x7f41f0103cc0 2024-05-17T04:30:12.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.246+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f41e4019070 con 0x7f41f0103cc0 2024-05-17T04:30:12.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.246+0000 7f41d7fff700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 0x7f41d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:12.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.246+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f41e40e1740 con 0x7f41f0103cc0 2024-05-17T04:30:12.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.246+0000 7f41eeffd700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 0x7f41d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:12.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.247+0000 7f41eeffd700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 0x7f41d80b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005950 tx=0x7f41e000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:12.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.249+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41e40a9ab0 con 0x7f41f0103cc0 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi105 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:12.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:12.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: osdmap e66: 11 total, 11 up, 11 in 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi105 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:12.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:12.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:12.461+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f41f002ea10 con 0x7f41d80b2880 2024-05-17T04:30:13.216 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 4272M 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:30:13.217 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 4272M 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:30:13.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:13.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 4272M 2024-05-17T04:30:13.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:13.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:13.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:13.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-17T04:30:13.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-17T04:30:13.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:14.244 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:13 smithi110 ceph-mon[35497]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:14.244 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:13 smithi110 ceph-mon[35497]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:13 smithi105 ceph-mon[34795]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:14.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:13 smithi105 ceph-mon[34795]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:14.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:13 smithi050 ceph-mon[31739]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:14.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:13 smithi050 ceph-mon[31739]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/2753036101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]': finished 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:15 smithi105 ceph-mon[34795]: from='client.? 172.21.15.110:0/3836191891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:30:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/2753036101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]': finished 2024-05-17T04:30:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T04:30:16.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:16.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:15 smithi110 ceph-mon[35497]: from='client.? 172.21.15.110:0/3836191891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/2753036101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]: dispatch 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ae7edc4-3891-4484-b0ca-d836265d6592"}]': finished 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: osdmap e67: 12 total, 11 up, 12 in 2024-05-17T04:30:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:16.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:15 smithi050 ceph-mon[31739]: from='client.? 172.21.15.110:0/3836191891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-17T04:30:18.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:17 smithi105 ceph-mon[34795]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-17T04:30:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:17 smithi110 ceph-mon[35497]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-17T04:30:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:17 smithi050 ceph-mon[31739]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-17T04:30:20.288 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:19 smithi110 ceph-mon[35497]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:30:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:19 smithi105 ceph-mon[34795]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:30:20.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:19 smithi050 ceph-mon[31739]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-17T04:30:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:21 smithi105 ceph-mon[34795]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T04:30:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:21 smithi110 ceph-mon[35497]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T04:30:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:21 smithi050 ceph-mon[31739]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-17T04:30:23.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T04:30:23.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T04:30:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:23.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-17T04:30:23.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:23 smithi105 ceph-mon[34795]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:23 smithi105 ceph-mon[34795]: Deploying daemon osd.11 on smithi110 2024-05-17T04:30:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:23 smithi110 ceph-mon[35497]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:23 smithi110 ceph-mon[35497]: Deploying daemon osd.11 on smithi110 2024-05-17T04:30:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:23 smithi050 ceph-mon[31739]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:23 smithi050 ceph-mon[31739]: Deploying daemon osd.11 on smithi110 2024-05-17T04:30:26.255 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:25 smithi110 ceph-mon[35497]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:26.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:25 smithi105 ceph-mon[34795]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:25 smithi050 ceph-mon[31739]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-17T04:30:28.230 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:28 smithi110 ceph-mon[35497]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:28 smithi105 ceph-mon[34795]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:27 smithi050 ceph-mon[31739]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:29 smithi050 ceph-mon[31739]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:30.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:29 smithi105 ceph-mon[34795]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:30.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:30.184 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:29 smithi110 ceph-mon[35497]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:30.184 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.184 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:30.185 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:32.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:31 smithi050 ceph-mon[31739]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:32.091 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:31 smithi110 ceph-mon[35497]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:31 smithi105 ceph-mon[34795]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 11 on host 'smithi110' 2024-05-17T04:30:32.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.781+0000 7f41d7fff700 1 -- 172.21.15.110:0/3741028664 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f41f002ea10 con 0x7f41d80b2880 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 msgr2=0x7f41d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 0x7f41d80b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f41e0005950 tx=0x7f41e000a300 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 msgr2=0x7f41f0199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41e400d950 tx=0x7f41e400dc60 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 shutdown_connections 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41f0104660 0x7f41f01a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f41d80b2880 0x7f41d80b4d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f41f0102b90 0x7f41f0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 --2- 172.21.15.110:0/3741028664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41f0103cc0 0x7f41f0199650 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 >> 172.21.15.110:0/3741028664 conn(0x7f41f00fe3c0 msgr2=0x7f41f0106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:32.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 shutdown_connections 2024-05-17T04:30:32.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:32.784+0000 7f41f535c700 1 -- 172.21.15.110:0/3741028664 wait complete. 2024-05-17T04:30:33.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:33.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:33.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:33.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:32 smithi110 ceph-mon[35497]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:33.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:32 smithi105 ceph-mon[34795]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:32 smithi050 ceph-mon[31739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-17T04:30:33.436 DEBUG:teuthology.orchestra.run.smithi110:osd.11> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.11.service 2024-05-17T04:30:33.438 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-17T04:30:33.438 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T04:30:33.702 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:34.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:33 smithi110 ceph-mon[35497]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T04:30:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:34.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T04:30:34.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:34.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:33 smithi050 ceph-mon[31739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: from='osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: osdmap e68: 12 total, 11 up, 12 in 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:33 smithi105 ceph-mon[34795]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-17T04:30:35.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:30:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:30:35.256+0000 7f44da84f700 -1 osd.11 0 waiting for initial osdmap 2024-05-17T04:30:35.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:30:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:30:35.271+0000 7f44d25fc700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 -- 172.21.15.50:0/2663554565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 msgr2=0x7fba28106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 --2- 172.21.15.50:0/2663554565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba28106bb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba18009990 tx=0x7fba18009ca0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 -- 172.21.15.50:0/2663554565 shutdown_connections 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 --2- 172.21.15.50:0/2663554565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 0x7fba2810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 --2- 172.21.15.50:0/2663554565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba28106bb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 --2- 172.21.15.50:0/2663554565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 0x7fba28109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 -- 172.21.15.50:0/2663554565 >> 172.21.15.50:0/2663554565 conn(0x7fba280fe3c0 msgr2=0x7fba28100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 -- 172.21.15.50:0/2663554565 shutdown_connections 2024-05-17T04:30:35.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.729+0000 7fba2e293700 1 -- 172.21.15.50:0/2663554565 wait complete. 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.730+0000 7fba2e293700 1 Processor -- start 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.730+0000 7fba2e293700 1 -- start start 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 0x7fba2819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 0x7fba281a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810f5c0 con 0x7fba28104790 2024-05-17T04:30:35.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810f440 con 0x7fba28109a50 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2e293700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810f740 con 0x7fba281070f0 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60254/0 (socket says 172.21.15.50:60254) 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba277fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 0x7fba2819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 -- 172.21.15.50:0/2011755681 learned_addr learned my addr 172.21.15.50:0/2011755681 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba2c830700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 0x7fba281a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 -- 172.21.15.50:0/2011755681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 msgr2=0x7fba2819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 0x7fba2819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 -- 172.21.15.50:0/2011755681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 msgr2=0x7fba281a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:35.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 0x7fba281a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 -- 172.21.15.50:0/2011755681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba281a1d40 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.731+0000 7fba27fff700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba18006010 tx=0x7fba1800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.732+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18026070 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.732+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba281a1fd0 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.732+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba1800f460 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.732+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800fe60 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.732+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba28102580 con 0x7fba28104790 2024-05-17T04:30:35.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.733+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba28107500 con 0x7fba28104790 2024-05-17T04:30:35.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.739+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fba18029070 con 0x7fba28104790 2024-05-17T04:30:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.740+0000 7fba257fa700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 0x7fba100b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.740+0000 7fba277fe700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 0x7fba100b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.740+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7fba180e2ce0 con 0x7fba28104790 2024-05-17T04:30:35.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.740+0000 7fba277fe700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 0x7fba100b9220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba2819b290 tx=0x7fba1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:35.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.743+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba18013070 con 0x7fba28104790 2024-05-17T04:30:35.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.958+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fba280647e0 con 0x7fba28104790 2024-05-17T04:30:35.958 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:35.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:30:35.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T04:30:35.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:35.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:35.959 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:35 smithi050 ceph-mon[31739]: from='osd.11 ' entity='osd.11' 2024-05-17T04:30:35.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.958+0000 7fba257fa700 1 -- 172.21.15.50:0/2011755681 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7fba180ab050 con 0x7fba28104790 2024-05-17T04:30:35.959 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:30:35.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 msgr2=0x7fba100b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:35.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 0x7fba100b9220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba2819b290 tx=0x7fba1c009260 comp rx=0 tx=0).stop 2024-05-17T04:30:35.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 msgr2=0x7fba2819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:35.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba18006010 tx=0x7fba1800bb40 comp rx=0 tx=0).stop 2024-05-17T04:30:35.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 shutdown_connections 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba28109a50 0x7fba281a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fba100b6d10 0x7fba100b9220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba28104790 0x7fba2819a1e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 --2- 172.21.15.50:0/2011755681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba281070f0 0x7fba2819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 >> 172.21.15.50:0/2011755681 conn(0x7fba280fe3c0 msgr2=0x7fba281054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 shutdown_connections 2024-05-17T04:30:35.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:35.962+0000 7fba2e293700 1 -- 172.21.15.50:0/2011755681 wait complete. 2024-05-17T04:30:36.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:36.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:30:36.025 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T04:30:36.026 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.026 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.026 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:35 smithi110 ceph-mon[35497]: from='osd.11 ' entity='osd.11' 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: osdmap e69: 12 total, 11 up, 12 in 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:35 smithi105 ceph-mon[34795]: from='osd.11 ' entity='osd.11' 2024-05-17T04:30:36.639 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1715920208,"num_in_osds":12,"osd_in_since":1715920215,"num_remapped_pgs":0} 2024-05-17T04:30:36.947 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: purged_snaps scrub starts 2024-05-17T04:30:36.947 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: purged_snaps scrub ok 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2011755681' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] boot 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:36.948 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:36.949 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: purged_snaps scrub starts 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: purged_snaps scrub ok 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2011755681' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] boot 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:37.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:37.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: purged_snaps scrub starts 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: purged_snaps scrub ok 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2011755681' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: osd.11 [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] boot 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: osdmap e70: 12 total, 12 up, 12 in 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-17T04:30:37.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:37.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:37.640 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-17T04:30:37.904 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:38.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:37 smithi050 ceph-mon[31739]: Detected new or changed devices on smithi110 2024-05-17T04:30:38.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:37 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-17T04:30:38.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:37 smithi050 ceph-mon[31739]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:38.027 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:37 smithi050 ceph-mon[31739]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T04:30:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:37 smithi105 ceph-mon[34795]: Detected new or changed devices on smithi110 2024-05-17T04:30:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:37 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-17T04:30:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:37 smithi105 ceph-mon[34795]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:38.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:37 smithi105 ceph-mon[34795]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T04:30:38.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:37 smithi110 ceph-mon[35497]: Detected new or changed devices on smithi110 2024-05-17T04:30:38.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:37 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-17T04:30:38.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:37 smithi110 ceph-mon[35497]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-17T04:30:38.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:37 smithi110 ceph-mon[35497]: osdmap e71: 12 total, 12 up, 12 in 2024-05-17T04:30:39.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:39 smithi050 ceph-mon[31739]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:39.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 -- 172.21.15.50:0/992780690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4102b90 msgr2=0x7ffac4102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/992780690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4102b90 0x7ffac4102fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b0d0 tx=0x7ffab400b3e0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 -- 172.21.15.50:0/992780690 shutdown_connections 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/992780690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4104660 0x7ffac410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/992780690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4102b90 0x7ffac4102fa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/992780690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 0x7ffac41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 -- 172.21.15.50:0/992780690 >> 172.21.15.50:0/992780690 conn(0x7ffac40fe3c0 msgr2=0x7ffac4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 -- 172.21.15.50:0/992780690 shutdown_connections 2024-05-17T04:30:39.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.832+0000 7ffac8fc5700 1 -- 172.21.15.50:0/992780690 wait complete. 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 Processor -- start 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 -- start start 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4102b90 0x7ffac406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 0x7ffac419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410e560 con 0x7ffac4104660 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410e3e0 con 0x7ffac4103cc0 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.833+0000 7ffac8fc5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410e6e0 con 0x7ffac4102b90 2024-05-17T04:30:39.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55250/0 (socket says 172.21.15.50:55250) 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4102b90 0x7ffac406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 -- 172.21.15.50:0/1294557846 learned_addr learned my addr 172.21.15.50:0/1294557846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac1d9b700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 0x7ffac419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 -- 172.21.15.50:0/1294557846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4102b90 msgr2=0x7ffac406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4102b90 0x7ffac406caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 -- 172.21.15.50:0/1294557846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 msgr2=0x7ffac419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 0x7ffac419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 -- 172.21.15.50:0/1294557846 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac41a1da0 con 0x7ffac4104660 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac2d9d700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d980 tx=0x7ffabc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc017070 con 0x7ffac4104660 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffabc005020 con 0x7ffac4104660 2024-05-17T04:30:39.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac41a2090 con 0x7ffac4104660 2024-05-17T04:30:39.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc009990 con 0x7ffac4104660 2024-05-17T04:30:39.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.834+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a2600 con 0x7ffac4104660 2024-05-17T04:30:39.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.836+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac4102fa0 con 0x7ffac4104660 2024-05-17T04:30:39.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.837+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffabc004a30 con 0x7ffac4104660 2024-05-17T04:30:39.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.838+0000 7ffabb7fe700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 0x7ffab00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:39.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.838+0000 7ffac259c700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 0x7ffab00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:39.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.838+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ffabc0e1b60 con 0x7ffac4104660 2024-05-17T04:30:39.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.838+0000 7ffac259c700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 0x7ffab00b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:39.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:39.842+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffabc020020 con 0x7ffac4104660 2024-05-17T04:30:40.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.057+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ffac4064690 con 0x7ffac4104660 2024-05-17T04:30:40.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.057+0000 7ffabb7fe700 1 -- 172.21.15.50:0/1294557846 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7ffabc0a9d20 con 0x7ffac4104660 2024-05-17T04:30:40.058 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.059+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 msgr2=0x7ffab00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.059+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 0x7ffab00b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab400b040 comp rx=0 tx=0).stop 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 msgr2=0x7ffac41a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d980 tx=0x7ffabc00dc90 comp rx=0 tx=0).stop 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 shutdown_connections 2024-05-17T04:30:40.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ffab00b2880 0x7ffab00b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffac4104660 0x7ffac41a1570 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffac4102b90 0x7ffac406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 --2- 172.21.15.50:0/1294557846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffac4103cc0 0x7ffac419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 >> 172.21.15.50:0/1294557846 conn(0x7ffac40fe3c0 msgr2=0x7ffac4106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 shutdown_connections 2024-05-17T04:30:40.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:40.060+0000 7ffac8fc5700 1 -- 172.21.15.50:0/1294557846 wait complete. 2024-05-17T04:30:40.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:39 smithi105 ceph-mon[34795]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:40.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:39 smithi110 ceph-mon[35497]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:40.856 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715920236,"num_in_osds":12,"osd_in_since":1715920215,"num_remapped_pgs":0} 2024-05-17T04:30:40.857 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:30:41.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:40 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1294557846' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:41.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:40 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1294557846' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:41.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:40 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1294557846' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-17T04:30:41.110 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:41.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:41 smithi050 ceph-mon[31739]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:42.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:41 smithi105 ceph-mon[34795]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:42.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:41 smithi110 ceph-mon[35497]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 -- 172.21.15.50:0/1421931639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 msgr2=0x7f6524107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 --2- 172.21.15.50:0/1421931639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f6524107b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514009ca0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 -- 172.21.15.50:0/1421931639 shutdown_connections 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 --2- 172.21.15.50:0/1421931639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 --2- 172.21.15.50:0/1421931639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 0x7f6524103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 --2- 172.21.15.50:0/1421931639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f6524107b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 -- 172.21.15.50:0/1421931639 >> 172.21.15.50:0/1421931639 conn(0x7f6524079af0 msgr2=0x7f6524079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 -- 172.21.15.50:0/1421931639 shutdown_connections 2024-05-17T04:30:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.900+0000 7f652b3c9700 1 -- 172.21.15.50:0/1421931639 wait complete. 2024-05-17T04:30:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 Processor -- start 2024-05-17T04:30:42.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 -- start start 2024-05-17T04:30:42.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 0x7f652419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6524112b60 con 0x7f6524107750 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65241129e0 con 0x7f65241080a0 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.901+0000 7f652b3c9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6524112ce0 con 0x7f6524102e20 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55264/0 (socket says 172.21.15.50:55264) 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 -- 172.21.15.50:0/3537833643 learned_addr learned my addr 172.21.15.50:0/3537833643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6529165700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 0x7f652419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6529966700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 -- 172.21.15.50:0/3537833643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 msgr2=0x7f652419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 0x7f652419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 -- 172.21.15.50:0/3537833643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 msgr2=0x7f652419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 -- 172.21.15.50:0/3537833643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652419f660 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6529966700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f6528964700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65140059d0 tx=0x7f651400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514027070 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652419f8f0 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6514022470 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514017650 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.902+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652419fda0 con 0x7f6524107750 2024-05-17T04:30:42.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.904+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6524103230 con 0x7f6524107750 2024-05-17T04:30:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.905+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6514009d10 con 0x7f6524107750 2024-05-17T04:30:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.906+0000 7f651a7fc700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 0x7f65100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.906+0000 7f6529165700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 0x7f65100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.906+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f65140e2a40 con 0x7f6524107750 2024-05-17T04:30:42.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.906+0000 7f6529165700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 0x7f65100b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6520005950 tx=0x7f65200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:42.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:42.910+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6514013070 con 0x7f6524107750 2024-05-17T04:30:43.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.128+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f652419baf0 con 0x7f6524107750 2024-05-17T04:30:43.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.129+0000 7f651a7fc700 1 -- 172.21.15.50:0/3537833643 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18431 (secure 0 0 0) 0x7f65140aac90 con 0x7f6524107750 2024-05-17T04:30:43.129 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:30:43.129 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":71,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:30:37.478291+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 msgr2=0x7f65100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 0x7f65100b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6520005950 tx=0x7f65200058e0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 msgr2=0x7f652419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65140059d0 tx=0x7f651400bfc0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 shutdown_connections 2024-05-17T04:30:43.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65241080a0 0x7f652419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524102e20 0x7f652419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f65100b2880 0x7f65100b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 --2- 172.21.15.50:0/3537833643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6524107750 0x7f652419ad30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 >> 172.21.15.50:0/3537833643 conn(0x7f6524079af0 msgr2=0x7f652406c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 shutdown_connections 2024-05-17T04:30:43.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:43.131+0000 7f652b3c9700 1 -- 172.21.15.50:0/3537833643 wait complete. 2024-05-17T04:30:43.804 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:43 smithi050 ceph-mon[31739]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:43.804 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:43 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3537833643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:30:43.837 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T04:26:42.688129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T04:30:43.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-17T04:30:44.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:43 smithi105 ceph-mon[34795]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:44.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:43 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3537833643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:30:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:43 smithi110 ceph-mon[35497]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:44.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:43 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3537833643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:30:44.090 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 -- 172.21.15.50:0/3271280378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8044104e90 msgr2=0x7f80441052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 --2- 172.21.15.50:0/3271280378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8044104e90 0x7f80441052a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8038009990 tx=0x7f8038009ca0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 -- 172.21.15.50:0/3271280378 shutdown_connections 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 --2- 172.21.15.50:0/3271280378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804410e270 0x7f8044110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 --2- 172.21.15.50:0/3271280378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8044104e90 0x7f80441052a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 --2- 172.21.15.50:0/3271280378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80441057e0 0x7f804410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 -- 172.21.15.50:0/3271280378 >> 172.21.15.50:0/3271280378 conn(0x7f8044079ae0 msgr2=0x7f8044079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:45.899 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:45 smithi050 ceph-mon[31739]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 -- 172.21.15.50:0/3271280378 shutdown_connections 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.897+0000 7f804af7f700 1 -- 172.21.15.50:0/3271280378 wait complete. 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.898+0000 7f804af7f700 1 Processor -- start 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.898+0000 7f804af7f700 1 -- start start 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 0x7f804419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f804410e270 0x7f804419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8044113c60 con 0x7f80441057e0 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8044113ae0 con 0x7f8044104e90 2024-05-17T04:30:45.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804af7f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8044113de0 con 0x7f804410e270 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55282/0 (socket says 172.21.15.50:55282) 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 -- 172.21.15.50:0/1709800945 learned_addr learned my addr 172.21.15.50:0/1709800945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f804951c700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f804410e270 0x7f804419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8048d1b700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 0x7f804419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 -- 172.21.15.50:0/1709800945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f804410e270 msgr2=0x7f804419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f804410e270 0x7f804419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 -- 172.21.15.50:0/1709800945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 msgr2=0x7f804419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 0x7f804419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8043fff700 1 -- 172.21.15.50:0/1709800945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804419f4c0 con 0x7f80441057e0 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.899+0000 7f8048d1b700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 0x7f804419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f8043fff700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f803000e9a0 tx=0x7f803000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:45.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803000cbd0 con 0x7f80441057e0 2024-05-17T04:30:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804419f7b0 con 0x7f80441057e0 2024-05-17T04:30:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f803000cd70 con 0x7f80441057e0 2024-05-17T04:30:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030018680 con 0x7f80441057e0 2024-05-17T04:30:45.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.900+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441a7180 con 0x7f80441057e0 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.902+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80441052a0 con 0x7f80441057e0 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.903+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8030010430 con 0x7f80441057e0 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.904+0000 7f8041ffb700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 0x7f80340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.904+0000 7f8048d1b700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 0x7f80340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.904+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f80300e2080 con 0x7f80441057e0 2024-05-17T04:30:45.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.904+0000 7f8048d1b700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 0x7f80340b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f803800b6f0 tx=0x7f803800b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:45.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:45.908+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80300aa1a0 con 0x7f80441057e0 2024-05-17T04:30:46.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:45 smithi105 ceph-mon[34795]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:46.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:45 smithi110 ceph-mon[35497]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:46.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.120+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f804419ba00 con 0x7f80441057e0 2024-05-17T04:30:46.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.121+0000 7f8041ffb700 1 -- 172.21.15.50:0/1709800945 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f80300b2220 con 0x7f80441057e0 2024-05-17T04:30:46.121 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-05-17T04:30:46.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 msgr2=0x7f80340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:46.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 0x7f80340b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f803800b6f0 tx=0x7f803800b2a0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 msgr2=0x7f804419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f803000e9a0 tx=0x7f803000ecb0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 shutdown_connections 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f804410e270 0x7f804419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8044104e90 0x7f804419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80340b2880 0x7f80340b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 --2- 172.21.15.50:0/1709800945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f80441057e0 0x7f804419ac40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 >> 172.21.15.50:0/1709800945 conn(0x7f8044079ae0 msgr2=0x7f80440ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 shutdown_connections 2024-05-17T04:30:46.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:46.123+0000 7f804af7f700 1 -- 172.21.15.50:0/1709800945 wait complete. 2024-05-17T04:30:46.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:46 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1709800945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:30:46.901 INFO:tasks.cephadm:Adding mds.a on smithi050 2024-05-17T04:30:46.901 INFO:tasks.cephadm:Adding mds.d on smithi050 2024-05-17T04:30:46.901 INFO:tasks.cephadm:Adding mds.g on smithi050 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.j on smithi050 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.b on smithi105 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.e on smithi105 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.h on smithi105 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.k on smithi105 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.c on smithi110 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.f on smithi110 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.i on smithi110 2024-05-17T04:30:46.902 INFO:tasks.cephadm:Adding mds.l on smithi110 2024-05-17T04:30:46.902 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l' 2024-05-17T04:30:47.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:46 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1709800945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:30:47.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:46 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1709800945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:30:47.154 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:30:48.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:47 smithi105 ceph-mon[34795]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:48.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:47 smithi110 ceph-mon[35497]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:47 smithi050 ceph-mon[31739]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:48.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- 172.21.15.110:0/467371219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 msgr2=0x7f2eb8102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:48.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 --2- 172.21.15.110:0/467371219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb8102fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0099c0 tx=0x7f2eac009cd0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- 172.21.15.110:0/467371219 shutdown_connections 2024-05-17T04:30:48.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 --2- 172.21.15.110:0/467371219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 0x7f2eb810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 --2- 172.21.15.110:0/467371219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb8102fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 --2- 172.21.15.110:0/467371219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb81040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- 172.21.15.110:0/467371219 >> 172.21.15.110:0/467371219 conn(0x7f2eb80fe3c0 msgr2=0x7f2eb8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- 172.21.15.110:0/467371219 shutdown_connections 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- 172.21.15.110:0/467371219 wait complete. 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 Processor -- start 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.788+0000 7f2ebf506700 1 -- start start 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 0x7f2eb81a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810e530 con 0x7f2eb8103cc0 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810e3b0 con 0x7f2eb8104660 2024-05-17T04:30:48.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebf506700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810e6b0 con 0x7f2eb8102b90 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50028/0 (socket says 172.21.15.110:50028) 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 -- 172.21.15.110:0/1635470116 learned_addr learned my addr 172.21.15.110:0/1635470116 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebdaa3700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 0x7f2eb81a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebcaa1700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 -- 172.21.15.110:0/1635470116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 msgr2=0x7f2eb81a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 0x7f2eb81a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 -- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 msgr2=0x7f2eb819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 -- 172.21.15.110:0/1635470116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb81a1dd0 con 0x7f2eb8102b90 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebcaa1700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2ebd2a2700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009990 tx=0x7f2eac00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:48.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.789+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac027070 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.790+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eac00bd30 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.790+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac022d30 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.790+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb8198e00 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.790+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb8199290 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.790+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb8102fa0 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.792+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2eac02a070 con 0x7f2eb8102b90 2024-05-17T04:30:48.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.793+0000 7f2eaa7fc700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 0x7f2ea40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:48.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.794+0000 7f2ebcaa1700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 0x7f2ea40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:48.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.794+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2eac0e3270 con 0x7f2eb8102b90 2024-05-17T04:30:48.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.794+0000 7f2ebcaa1700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 0x7f2ea40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806c450 tx=0x7f2eb4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:48.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:48.797+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2eac013070 con 0x7f2eb8102b90 2024-05-17T04:30:49.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.010+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}) -- 0x7f2eb819ac20 con 0x7f2ea40b2880 2024-05-17T04:30:49.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.019+0000 7f2eaa7fc700 1 -- 172.21.15.110:0/1635470116 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2eb819ac20 con 0x7f2ea40b2880 2024-05-17T04:30:49.020 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mds.all update... 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 msgr2=0x7f2ea40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 0x7f2ea40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2eb806c450 tx=0x7f2eb4008040 comp rx=0 tx=0).stop 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 msgr2=0x7f2eb806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009990 tx=0x7f2eac00f690 comp rx=0 tx=0).stop 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 shutdown_connections 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eb8104660 0x7f2eb81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:49.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2eb8102b90 0x7f2eb806caa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ea40b2880 0x7f2ea40b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 --2- 172.21.15.110:0/1635470116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2eb8103cc0 0x7f2eb819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.021+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 >> 172.21.15.110:0/1635470116 conn(0x7f2eb80fe3c0 msgr2=0x7f2eb8106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.022+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 shutdown_connections 2024-05-17T04:30:49.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-17T04:30:49.022+0000 7f2ebf506700 1 -- 172.21.15.110:0/1635470116 wait complete. 2024-05-17T04:30:49.689 DEBUG:teuthology.orchestra.run.smithi050:mds.a> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.a.service 2024-05-17T04:30:49.692 DEBUG:teuthology.orchestra.run.smithi050:mds.d> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.d.service 2024-05-17T04:30:49.735 DEBUG:teuthology.orchestra.run.smithi050:mds.g> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.g.service 2024-05-17T04:30:49.778 DEBUG:teuthology.orchestra.run.smithi050:mds.j> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.j.service 2024-05-17T04:30:49.821 DEBUG:teuthology.orchestra.run.smithi105:mds.b> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.b.service 2024-05-17T04:30:49.824 DEBUG:teuthology.orchestra.run.smithi105:mds.e> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.e.service 2024-05-17T04:30:49.867 DEBUG:teuthology.orchestra.run.smithi105:mds.h> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.h.service 2024-05-17T04:30:49.910 DEBUG:teuthology.orchestra.run.smithi105:mds.k> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.k.service 2024-05-17T04:30:49.953 DEBUG:teuthology.orchestra.run.smithi110:mds.c> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.c.service 2024-05-17T04:30:49.956 DEBUG:teuthology.orchestra.run.smithi110:mds.f> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.f.service 2024-05-17T04:30:49.999 DEBUG:teuthology.orchestra.run.smithi110:mds.i> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.i.service 2024-05-17T04:30:50.043 DEBUG:teuthology.orchestra.run.smithi110:mds.l> sudo journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.l.service 2024-05-17T04:30:50.086 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-17T04:30:50.087 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:30:50.282 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:50.282 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:50.282 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:50.283 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: Saving service mds.all spec with placement smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:30:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:30:50.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:50.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:50.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:50.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:50.355 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:51.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:51 smithi105 ceph-mon[34795]: Deploying daemon mds.l on smithi110 2024-05-17T04:30:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:51 smithi110 ceph-mon[35497]: Deploying daemon mds.l on smithi110 2024-05-17T04:30:51.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:51 smithi050 ceph-mon[31739]: Deploying daemon mds.l on smithi110 2024-05-17T04:30:52.024 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:51 smithi110 systemd[1]: Starting Ceph mds.l for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:30:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 -- 172.21.15.50:0/3948525403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 msgr2=0x7f7ae4107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/3948525403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae4107b60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009990 tx=0x7f7ad8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 -- 172.21.15.50:0/3948525403 shutdown_connections 2024-05-17T04:30:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/3948525403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae41080a0 0x7f7ae410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/3948525403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae4102e20 0x7f7ae4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/3948525403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae4107b60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 -- 172.21.15.50:0/3948525403 >> 172.21.15.50:0/3948525403 conn(0x7f7ae4079af0 msgr2=0x7f7ae4079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 -- 172.21.15.50:0/3948525403 shutdown_connections 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.126+0000 7f7ae9db5700 1 -- 172.21.15.50:0/3948525403 wait complete. 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 Processor -- start 2024-05-17T04:30:52.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 -- start start 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae4102e20 0x7f7ae419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae41080a0 0x7f7ae419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae4112b60 con 0x7f7ae4107750 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae41129e0 con 0x7f7ae4102e20 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.127+0000 7f7ae9db5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae4112ce0 con 0x7f7ae41080a0 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52002/0 (socket says 172.21.15.50:52002) 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 -- 172.21.15.50:0/37430378 learned_addr learned my addr 172.21.15.50:0/37430378 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae37fe700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae4102e20 0x7f7ae419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:52.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae3fff700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae41080a0 0x7f7ae419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 -- 172.21.15.50:0/37430378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae41080a0 msgr2=0x7f7ae419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae41080a0 0x7f7ae419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 -- 172.21.15.50:0/37430378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae4102e20 msgr2=0x7f7ae419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae4102e20 0x7f7ae419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 -- 172.21.15.50:0/37430378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae419f570 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae2ffd700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000c00 tx=0x7f7ad80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8027070 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ad800fb80 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae419f800 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8022d00 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.128+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae419fce0 con 0x7f7ae4107750 2024-05-17T04:30:52.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.129+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae4103230 con 0x7f7ae4107750 2024-05-17T04:30:52.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.131+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7ad802a070 con 0x7f7ae4107750 2024-05-17T04:30:52.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.132+0000 7f7ae0ff9700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 0x7f7acc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:52.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.132+0000 7f7ae37fe700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 0x7f7acc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:52.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.132+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7ad80e3060 con 0x7f7ae4107750 2024-05-17T04:30:52.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.132+0000 7f7ae37fe700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 0x7f7acc0b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005950 tx=0x7f7ad400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:52.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.135+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ad8031080 con 0x7f7ae4107750 2024-05-17T04:30:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:52 smithi110 ceph-mon[35497]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:52.307 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 podman[65719]: 2024-05-17 04:30:51.982920497 +0000 UTC m=+0.017846936 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:30:52.307 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 podman[65719]: 2024-05-17 04:30:52.096127591 +0000 UTC m=+0.131054037 container create fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-17T04:30:52.307 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 podman[65719]: 2024-05-17 04:30:52.307344109 +0000 UTC m=+0.342270530 container init fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:30:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:52 smithi105 ceph-mon[34795]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:52 smithi050 ceph-mon[31739]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:52.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.401+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ae419bab0 con 0x7f7ae4107750 2024-05-17T04:30:52.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.402+0000 7f7ae0ff9700 1 -- 172.21.15.50:0/37430378 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f7ad8013070 con 0x7f7ae4107750 2024-05-17T04:30:52.403 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:30:52.403 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T04:30:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 msgr2=0x7f7acc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 0x7f7acc0b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005950 tx=0x7f7ad400b410 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 msgr2=0x7f7ae419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000c00 tx=0x7f7ad80047b0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 shutdown_connections 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ae41080a0 0x7f7ae419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ae4102e20 0x7f7ae419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7acc0b2880 0x7f7acc0b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 --2- 172.21.15.50:0/37430378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7ae4107750 0x7f7ae419acf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 >> 172.21.15.50:0/37430378 conn(0x7f7ae4079af0 msgr2=0x7f7ae406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 shutdown_connections 2024-05-17T04:30:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:52.404+0000 7f7ae9db5700 1 -- 172.21.15.50:0/37430378 wait complete. 2024-05-17T04:30:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 2 2024-05-17T04:30:52.567 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 podman[65719]: 2024-05-17 04:30:52.310235431 +0000 UTC m=+0.345161866 container start fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-17T04:30:52.567 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 ceph-mds[65738]: -- 172.21.15.110:0/2592600775 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb2ade5a40 con 0x55eb2ae2ba80 2024-05-17T04:30:52.567 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l[65734]: starting mds.l at 2024-05-17T04:30:52.567 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 bash[65719]: fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 2024-05-17T04:30:52.567 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:30:52 smithi110 systemd[1]: Started Ceph mds.l for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6832/3447375489,v1:172.21.15.110:6833/3447375489] up:boot 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: fsmap 1 up:standby 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/37430378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.431 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:53.432 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:53.432 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:53.432 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: Deploying daemon mds.k on smithi105 2024-05-17T04:30:53.432 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:53 smithi105 ceph-mon[34795]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6832/3447375489,v1:172.21.15.110:6833/3447375489] up:boot 2024-05-17T04:30:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: fsmap 1 up:standby 2024-05-17T04:30:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T04:30:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/37430378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: Deploying daemon mds.k on smithi105 2024-05-17T04:30:53.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:53 smithi050 ceph-mon[31739]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6832/3447375489,v1:172.21.15.110:6833/3447375489] up:boot 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: fsmap 1 up:standby 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/37430378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: Deploying daemon mds.k on smithi105 2024-05-17T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:53 smithi110 ceph-mon[35497]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:54.815 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:54 smithi105 systemd[1]: Starting Ceph mds.k for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:30:55.225 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:54 smithi105 podman[63999]: 2024-05-17 04:30:54.790710378 +0000 UTC m=+0.017291191 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:30:55.225 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:54 smithi105 podman[63999]: 2024-05-17 04:30:54.980358729 +0000 UTC m=+0.206939587 container create fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:30:55.254 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:30:55.530 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 podman[63999]: 2024-05-17 04:30:55.225162909 +0000 UTC m=+0.451743724 container init fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 podman[63999]: 2024-05-17 04:30:55.228288573 +0000 UTC m=+0.454869386 container start fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mds[64019]: -- 172.21.15.105:0/524704770 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641c33a3a40 con 0x5641c33e9600 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k[64015]: starting mds.k at 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 bash[63999]: fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 2024-05-17T04:30:55.565 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:30:55 smithi105 systemd[1]: Started Ceph mds.k for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:30:56.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:56.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.022 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:56.022 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:56.022 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:56.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:30:56.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:30:56.939 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:56 smithi050 ceph-mon[31739]: Deploying daemon mds.j on smithi050 2024-05-17T04:30:56.939 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:56 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6832/32414548,v1:172.21.15.105:6833/32414548] up:boot 2024-05-17T04:30:56.939 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:56 smithi050 ceph-mon[31739]: fsmap 2 up:standby 2024-05-17T04:30:56.940 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T04:30:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:56 smithi105 ceph-mon[34795]: Deploying daemon mds.j on smithi050 2024-05-17T04:30:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:56 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6832/32414548,v1:172.21.15.105:6833/32414548] up:boot 2024-05-17T04:30:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:56 smithi105 ceph-mon[34795]: fsmap 2 up:standby 2024-05-17T04:30:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T04:30:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:56 smithi110 ceph-mon[35497]: Deploying daemon mds.j on smithi050 2024-05-17T04:30:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:56 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6832/32414548,v1:172.21.15.105:6833/32414548] up:boot 2024-05-17T04:30:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:56 smithi110 ceph-mon[35497]: fsmap 2 up:standby 2024-05-17T04:30:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 -- 172.21.15.50:0/1408335183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c103f20 msgr2=0x7fbe2c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 --2- 172.21.15.50:0/1408335183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c103f20 0x7fbe2c104330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009990 tx=0x7fbe28009ca0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 -- 172.21.15.50:0/1408335183 shutdown_connections 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 --2- 172.21.15.50:0/1408335183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 --2- 172.21.15.50:0/1408335183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c103f20 0x7fbe2c104330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 --2- 172.21.15.50:0/1408335183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c108850 0x7fbe2c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 -- 172.21.15.50:0/1408335183 >> 172.21.15.50:0/1408335183 conn(0x7fbe2c0ff9c0 msgr2=0x7fbe2c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 -- 172.21.15.50:0/1408335183 shutdown_connections 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.650+0000 7fbe33455700 1 -- 172.21.15.50:0/1408335183 wait complete. 2024-05-17T04:30:57.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 Processor -- start 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 -- start start 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c103f20 0x7fbe2c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c113c80 con 0x7fbe2c108850 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c113b00 con 0x7fbe2c103f20 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe33455700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c113e00 con 0x7fbe2c1091a0 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52016/0 (socket says 172.21.15.50:52016) 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 -- 172.21.15.50:0/3049596069 learned_addr learned my addr 172.21.15.50:0/3049596069 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe311f1700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c103f20 0x7fbe2c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe319f2700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:57.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 -- 172.21.15.50:0/3049596069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 msgr2=0x7fbe2c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 -- 172.21.15.50:0/3049596069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c103f20 msgr2=0x7fbe2c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c103f20 0x7fbe2c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe309f0700 1 -- 172.21.15.50:0/3049596069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c19f570 con 0x7fbe2c108850 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.651+0000 7fbe319f2700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe309f0700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00e970 tx=0x7fbe1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00cbd0 con 0x7fbe2c108850 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe1c00cd70 con 0x7fbe2c108850 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c19f860 con 0x7fbe2c108850 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c005570 con 0x7fbe2c108850 2024-05-17T04:30:57.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.652+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c1a7180 con 0x7fbe2c108850 2024-05-17T04:30:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.653+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2c104330 con 0x7fbe2c108850 2024-05-17T04:30:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.654+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbe1c004750 con 0x7fbe2c108850 2024-05-17T04:30:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.655+0000 7fbe227fc700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 0x7fbe180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:30:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.655+0000 7fbe311f1700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 0x7fbe180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:30:57.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.655+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fbe1c0e2020 con 0x7fbe2c108850 2024-05-17T04:30:57.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.656+0000 7fbe311f1700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 0x7fbe180b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800b2e0 tx=0x7fbe28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:30:57.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.660+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe1c0aa140 con 0x7fbe2c108850 2024-05-17T04:30:57.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:57 smithi050 ceph-mon[31739]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:57.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.928+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe2c02ea80 con 0x7fbe2c108850 2024-05-17T04:30:57.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.929+0000 7fbe227fc700 1 -- 172.21.15.50:0/3049596069 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fbe1c0a9a40 con 0x7fbe2c108850 2024-05-17T04:30:57.930 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:30:57.930 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T04:30:57.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 msgr2=0x7fbe180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 0x7fbe180b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800b2e0 tx=0x7fbe28005fb0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 msgr2=0x7fbe2c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00e970 tx=0x7fbe1c00ec80 comp rx=0 tx=0).stop 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 shutdown_connections 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe2c1091a0 0x7fbe2c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe2c103f20 0x7fbe2c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbe180b2900 0x7fbe180b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 --2- 172.21.15.50:0/3049596069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe2c108850 0x7fbe2c19ac40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 >> 172.21.15.50:0/3049596069 conn(0x7fbe2c0ff9c0 msgr2=0x7fbe2c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 shutdown_connections 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:30:57.931+0000 7fbe33455700 1 -- 172.21.15.50:0/3049596069 wait complete. 2024-05-17T04:30:57.935 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 3 2024-05-17T04:30:58.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:57 smithi105 ceph-mon[34795]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:57 smithi110 ceph-mon[35497]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:59.019 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:58 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3049596069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:59.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:58 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3049596069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:59.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:58 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3049596069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:30:59.576 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:30:59 smithi050 systemd[1]: Starting Ceph mds.j for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:30:59.974 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:30:59 smithi050 ceph-mon[31739]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:30:59.975 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:30:59 smithi050 podman[69287]: 2024-05-17 04:30:59.617485327 +0000 UTC m=+0.018080050 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:30:59.975 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:30:59 smithi050 podman[69287]: 2024-05-17 04:30:59.730822771 +0000 UTC m=+0.131417477 container create 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:30:59.975 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:30:59 smithi050 podman[69287]: 2024-05-17 04:30:59.975258482 +0000 UTC m=+0.375853199 container init 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:31:00.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:30:59 smithi105 ceph-mon[34795]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:00.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:30:59 smithi110 ceph-mon[35497]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:00.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:30:59 smithi050 podman[69287]: 2024-05-17 04:30:59.97837944 +0000 UTC m=+0.378974156 container start 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-17T04:31:00.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:31:00 smithi050 ceph-mds[69307]: -- 172.21.15.50:0/3180862103 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55782b81fa40 con 0x55782b865180 2024-05-17T04:31:00.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:31:00 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j[69303]: starting mds.j at 2024-05-17T04:31:00.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:31:00 smithi050 bash[69287]: 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 2024-05-17T04:31:00.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:31:00 smithi050 systemd[1]: Started Ceph mds.j for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:00.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:01.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: Deploying daemon mds.d on smithi050 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.50:6834/1993874157,v1:172.21.15.50:6835/1993874157] up:boot 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: fsmap 3 up:standby 2024-05-17T04:31:01.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:01 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: Deploying daemon mds.d on smithi050 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.50:6834/1993874157,v1:172.21.15.50:6835/1993874157] up:boot 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: fsmap 3 up:standby 2024-05-17T04:31:01.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:01 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: Deploying daemon mds.d on smithi050 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.50:6834/1993874157,v1:172.21.15.50:6835/1993874157] up:boot 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: fsmap 3 up:standby 2024-05-17T04:31:01.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:01 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-17T04:31:01.832 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:02.238 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:02 smithi050 ceph-mon[31739]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:02 smithi105 ceph-mon[34795]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:02.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:02 smithi110 ceph-mon[35497]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 -- 172.21.15.50:0/3734407255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc981080a0 msgr2=0x7efc9810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 --2- 172.21.15.50:0/3734407255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc981080a0 0x7efc9810fa00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc9000b0a0 tx=0x7efc9000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 -- 172.21.15.50:0/3734407255 shutdown_connections 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 --2- 172.21.15.50:0/3734407255 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc981080a0 0x7efc9810fa00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 --2- 172.21.15.50:0/3734407255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc98102e70 0x7efc98103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 --2- 172.21.15.50:0/3734407255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc98107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 -- 172.21.15.50:0/3734407255 >> 172.21.15.50:0/3734407255 conn(0x7efc98079ae0 msgr2=0x7efc98079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 -- 172.21.15.50:0/3734407255 shutdown_connections 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 -- 172.21.15.50:0/3734407255 wait complete. 2024-05-17T04:31:03.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.793+0000 7efc9e392700 1 Processor -- start 2024-05-17T04:31:03.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 -- start start 2024-05-17T04:31:03.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 0x7efc98105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:03.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc981056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc981080a0 0x7efc98105c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc98112b90 con 0x7efc98102e70 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc98112a10 con 0x7efc981080a0 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9e392700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc98112d10 con 0x7efc98107750 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 0x7efc98105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc97fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 0x7efc98105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41036/0 (socket says 172.21.15.50:41036) 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc97fff700 1 -- 172.21.15.50:0/3983428933 learned_addr learned my addr 172.21.15.50:0/3983428933 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc977fe700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc981056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.794+0000 7efc9c92f700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc981080a0 0x7efc98105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 -- 172.21.15.50:0/3983428933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc981080a0 msgr2=0x7efc98105c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc981080a0 0x7efc98105c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 -- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 msgr2=0x7efc98105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 0x7efc98105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 -- 172.21.15.50:0/3983428933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc981a8b40 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc977fe700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc981056d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d720 tx=0x7efc8c00da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c016070 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc8c004830 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc981a8dd0 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.795+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c005480 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.796+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981a9230 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.796+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc98103280 con 0x7efc98107750 2024-05-17T04:31:03.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.799+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc8c004220 con 0x7efc98107750 2024-05-17T04:31:03.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.799+0000 7efc957fa700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:03.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.799+0000 7efc97fff700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:03.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.799+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7efc8c0e1ff0 con 0x7efc98107750 2024-05-17T04:31:03.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.800+0000 7efc97fff700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 0x7efc800b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc88005950 tx=0x7efc88004080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:03.802+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc8c01f020 con 0x7efc98107750 2024-05-17T04:31:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.069+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc981066c0 con 0x7efc98107750 2024-05-17T04:31:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.070+0000 7efc957fa700 1 -- 172.21.15.50:0/3983428933 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7efc8c0aa110 con 0x7efc98107750 2024-05-17T04:31:04.071 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:04.071 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T04:31:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 msgr2=0x7efc800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 0x7efc800b4e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc88005950 tx=0x7efc88004080 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 msgr2=0x7efc981056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc981056d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d720 tx=0x7efc8c00da30 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 shutdown_connections 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc981080a0 0x7efc98105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc98102e70 0x7efc98105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 --2- 172.21.15.50:0/3983428933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc98107750 0x7efc981056d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 >> 172.21.15.50:0/3983428933 conn(0x7efc98079ae0 msgr2=0x7efc9806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 shutdown_connections 2024-05-17T04:31:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:04.072+0000 7efc9e392700 1 -- 172.21.15.50:0/3983428933 wait complete. 2024-05-17T04:31:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 4 2024-05-17T04:31:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:04 smithi050 ceph-mon[31739]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:04 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3983428933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:04 smithi105 ceph-mon[34795]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:04.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:04 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3983428933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:04 smithi110 ceph-mon[35497]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:04 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3983428933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:05.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.50:6836/2083526209,v1:172.21.15.50:6837/2083526209] up:boot 2024-05-17T04:31:05.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: fsmap 4 up:standby 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: Deploying daemon mds.e on smithi105 2024-05-17T04:31:05.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:05 smithi105 ceph-mon[34795]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.50:6836/2083526209,v1:172.21.15.50:6837/2083526209] up:boot 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: fsmap 4 up:standby 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: Deploying daemon mds.e on smithi105 2024-05-17T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:05 smithi110 ceph-mon[35497]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.50:6836/2083526209,v1:172.21.15.50:6837/2083526209] up:boot 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: fsmap 4 up:standby 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:05.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:05.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:05.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: Deploying daemon mds.e on smithi105 2024-05-17T04:31:05.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:05 smithi050 ceph-mon[31739]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:06.565 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 systemd[1]: Starting Ceph mds.e for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:06.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:06.919 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 podman[64314]: 2024-05-17 04:31:06.550906206 +0000 UTC m=+0.017203512 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:06.920 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 podman[64314]: 2024-05-17 04:31:06.654973493 +0000 UTC m=+0.121270799 container create f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-17T04:31:06.920 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 podman[64314]: 2024-05-17 04:31:06.883415305 +0000 UTC m=+0.349712605 container init f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:31:06.920 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 podman[64314]: 2024-05-17 04:31:06.886155843 +0000 UTC m=+0.352453131 container start f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:31:06.920 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 ceph-mds[64334]: -- 172.21.15.105:0/2000409082 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55737bddda40 con 0x55737be23180 2024-05-17T04:31:07.034 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:07.315 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: starting mds.e at 2024-05-17T04:31:07.315 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 bash[64314]: f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a 2024-05-17T04:31:07.315 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:31:06 smithi105 systemd[1]: Started Ceph mds.e for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: Deploying daemon mds.f on smithi110 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6834/1830970686,v1:172.21.15.105:6835/1830970686] up:boot 2024-05-17T04:31:08.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: fsmap 5 up:standby 2024-05-17T04:31:08.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:08 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: Deploying daemon mds.f on smithi110 2024-05-17T04:31:08.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6834/1830970686,v1:172.21.15.105:6835/1830970686] up:boot 2024-05-17T04:31:08.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: fsmap 5 up:standby 2024-05-17T04:31:08.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:08 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:31:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: Deploying daemon mds.f on smithi110 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6834/1830970686,v1:172.21.15.105:6835/1830970686] up:boot 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: fsmap 5 up:standby 2024-05-17T04:31:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:08 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1034108245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 msgr2=0x7fec88104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1034108245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec88104330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec78009990 tx=0x7fec78009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1034108245 shutdown_connections 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1034108245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec881091a0 0x7fec88110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1034108245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec88104330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1034108245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec88108850 0x7fec88108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1034108245 >> 172.21.15.50:0/1034108245 conn(0x7fec880ff9c0 msgr2=0x7fec88101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1034108245 shutdown_connections 2024-05-17T04:31:08.730 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.729+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1034108245 wait complete. 2024-05-17T04:31:08.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 Processor -- start 2024-05-17T04:31:08.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 -- start start 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec88108850 0x7fec8819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 0x7fec8819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec88113c80 con 0x7fec88103f20 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec88113b00 con 0x7fec88108850 2024-05-17T04:31:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8e9e5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec88113e00 con 0x7fec881091a0 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec87fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec87fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35004/0 (socket says 172.21.15.50:35004) 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec87fff700 1 -- 172.21.15.50:0/1771451041 learned_addr learned my addr 172.21.15.50:0/1771451041 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec877fe700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec88108850 0x7fec8819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.730+0000 7fec8cf82700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 0x7fec8819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 -- 172.21.15.50:0/1771451041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 msgr2=0x7fec8819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 0x7fec8819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 -- 172.21.15.50:0/1771451041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec88108850 msgr2=0x7fec8819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec88108850 0x7fec8819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 -- 172.21.15.50:0/1771451041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8819f5b0 con 0x7fec88103f20 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec8cf82700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 0x7fec8819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec87fff700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fec78000c00 tx=0x7fec78009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78027070 con 0x7fec88103f20 2024-05-17T04:31:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec7800fc70 con 0x7fec88103f20 2024-05-17T04:31:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8819f840 con 0x7fec88103f20 2024-05-17T04:31:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78022d40 con 0x7fec88103f20 2024-05-17T04:31:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.731+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec881a71c0 con 0x7fec88103f20 2024-05-17T04:31:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.733+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec881053d0 con 0x7fec88103f20 2024-05-17T04:31:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.734+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fec7802a070 con 0x7fec88103f20 2024-05-17T04:31:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.735+0000 7fec857fa700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 0x7fec700b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.735+0000 7fec877fe700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 0x7fec700b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.735+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fec780e2b50 con 0x7fec88103f20 2024-05-17T04:31:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.736+0000 7fec877fe700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 0x7fec700b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005950 tx=0x7fec7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:08.739+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec7805fdf0 con 0x7fec88103f20 2024-05-17T04:31:09.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.003+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec88104330 con 0x7fec88103f20 2024-05-17T04:31:09.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.003+0000 7fec857fa700 1 -- 172.21.15.50:0/1771451041 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7fec78013070 con 0x7fec88103f20 2024-05-17T04:31:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:09.004 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 msgr2=0x7fec700b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 0x7fec700b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005950 tx=0x7fec7c0058e0 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 msgr2=0x7fec8819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fec78000c00 tx=0x7fec78009f90 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 shutdown_connections 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec881091a0 0x7fec8819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec700b2900 0x7fec700b4e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec88103f20 0x7fec8819a6e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 --2- 172.21.15.50:0/1771451041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec88108850 0x7fec8819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 >> 172.21.15.50:0/1771451041 conn(0x7fec880ff9c0 msgr2=0x7fec881001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 shutdown_connections 2024-05-17T04:31:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:09.006+0000 7fec8e9e5700 1 -- 172.21.15.50:0/1771451041 wait complete. 2024-05-17T04:31:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 6 2024-05-17T04:31:09.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:09 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1771451041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:09.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:09 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1771451041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:09.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:09 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1771451041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:09.317 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 systemd[1]: Starting Ceph mds.f for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:09.786 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 podman[66033]: 2024-05-17 04:31:09.409772951 +0000 UTC m=+0.017078608 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:09.786 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 podman[66033]: 2024-05-17 04:31:09.542184453 +0000 UTC m=+0.149490148 container create f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:31:10.049 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 podman[66033]: 2024-05-17 04:31:09.786630128 +0000 UTC m=+0.393935780 container init f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 podman[66033]: 2024-05-17 04:31:09.789405368 +0000 UTC m=+0.396711026 container start f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 ceph-mds[66052]: -- 172.21.15.110:0/2578416566 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55791457fa40 con 0x5579145c5600 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: starting mds.f at 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 bash[66033]: f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:31:09 smithi110 systemd[1]: Started Ceph mds.f for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.050 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:10.302 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:10.302 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:10.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:10.303 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:10 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:10 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:10 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:11.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:11 smithi105 ceph-mon[34795]: Deploying daemon mds.c on smithi110 2024-05-17T04:31:11.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:11 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6834/1276100312,v1:172.21.15.110:6835/1276100312] up:boot 2024-05-17T04:31:11.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:11 smithi105 ceph-mon[34795]: fsmap 6 up:standby 2024-05-17T04:31:11.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:11 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:31:11.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:11 smithi110 ceph-mon[35497]: Deploying daemon mds.c on smithi110 2024-05-17T04:31:11.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:11 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6834/1276100312,v1:172.21.15.110:6835/1276100312] up:boot 2024-05-17T04:31:11.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:11 smithi110 ceph-mon[35497]: fsmap 6 up:standby 2024-05-17T04:31:11.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:11 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:31:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:11 smithi050 ceph-mon[31739]: Deploying daemon mds.c on smithi110 2024-05-17T04:31:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:11 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6834/1276100312,v1:172.21.15.110:6835/1276100312] up:boot 2024-05-17T04:31:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:11 smithi050 ceph-mon[31739]: fsmap 6 up:standby 2024-05-17T04:31:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:11 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:31:11.688 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:11.955 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:12.296 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:12 smithi110 ceph-mon[35497]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:12.296 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 systemd[1]: Starting Ceph mds.c for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:12 smithi105 ceph-mon[34795]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:12 smithi050 ceph-mon[31739]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:12.566 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 podman[66333]: 2024-05-17 04:31:12.199347581 +0000 UTC m=+0.017392452 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:12.567 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 podman[66333]: 2024-05-17 04:31:12.297005456 +0000 UTC m=+0.115050309 container create 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T04:31:12.567 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 podman[66333]: 2024-05-17 04:31:12.56700815 +0000 UTC m=+0.385053013 container init 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:31:12.821 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 podman[66333]: 2024-05-17 04:31:12.570008952 +0000 UTC m=+0.388053815 container start 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T04:31:12.822 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 ceph-mds[66352]: -- 172.21.15.110:0/2686106635 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bcba8b5a40 con 0x55bcba8fb180 2024-05-17T04:31:12.822 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: starting mds.c at 2024-05-17T04:31:12.822 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 bash[66333]: 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 2024-05-17T04:31:12.822 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:31:12 smithi110 systemd[1]: Started Ceph mds.c for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:13.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 -- 172.21.15.50:0/787145031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c1080a0 msgr2=0x7fe04c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:13.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 --2- 172.21.15.50:0/787145031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c1080a0 0x7fe04c10fa00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b0a0 tx=0x7fe04400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 -- 172.21.15.50:0/787145031 shutdown_connections 2024-05-17T04:31:13.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 --2- 172.21.15.50:0/787145031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c1080a0 0x7fe04c10fa00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 --2- 172.21.15.50:0/787145031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 --2- 172.21.15.50:0/787145031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c107750 0x7fe04c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 -- 172.21.15.50:0/787145031 >> 172.21.15.50:0/787145031 conn(0x7fe04c079ae0 msgr2=0x7fe04c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:13.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 -- 172.21.15.50:0/787145031 shutdown_connections 2024-05-17T04:31:13.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.688+0000 7fe05269f700 1 -- 172.21.15.50:0/787145031 wait complete. 2024-05-17T04:31:13.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 Processor -- start 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 -- start start 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c1080a0 0x7fe04c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c112b90 con 0x7fe04c107750 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c112a10 con 0x7fe04c102e70 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe05269f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c112d10 con 0x7fe04c1080a0 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe043fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe04bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:13.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe043fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35026/0 (socket says 172.21.15.50:35026) 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe04bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36016/0 (socket says 172.21.15.50:36016) 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe050c3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c1080a0 0x7fe04c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.689+0000 7fe043fff700 1 -- 172.21.15.50:0/3068527765 learned_addr learned my addr 172.21.15.50:0/3068527765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 -- 172.21.15.50:0/3068527765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c1080a0 msgr2=0x7fe04c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c1080a0 0x7fe04c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 -- 172.21.15.50:0/3068527765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 msgr2=0x7fe04c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 -- 172.21.15.50:0/3068527765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c19f670 con 0x7fe04c107750 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe043fff700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe03400d950 tx=0x7fe03400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034016070 con 0x7fe04c107750 2024-05-17T04:31:13.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c19f900 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe034004830 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034005420 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.690+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c1a7280 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.691+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04c103280 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.693+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe034004220 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.694+0000 7fe049ffb700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 0x7fe0380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.694+0000 7fe04bfff700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 0x7fe0380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.694+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe0340e1a00 con 0x7fe04c107750 2024-05-17T04:31:13.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.694+0000 7fe04bfff700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 0x7fe0380b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009c60 tx=0x7fe03c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:13.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.698+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe03401f020 con 0x7fe04c107750 2024-05-17T04:31:13.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.961+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe04c02ea80 con 0x7fe04c107750 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: Deploying daemon mds.h on smithi105 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:boot 2024-05-17T04:31:13.963 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: fsmap 7 up:standby 2024-05-17T04:31:13.964 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:13 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:31:13.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.962+0000 7fe049ffb700 1 -- 172.21.15.50:0/3068527765 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7fe0340a9ba0 con 0x7fe04c107750 2024-05-17T04:31:13.964 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:13.964 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-17T04:31:13.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 msgr2=0x7fe0380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 0x7fe0380b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009c60 tx=0x7fe03c009260 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 msgr2=0x7fe04c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe03400d950 tx=0x7fe03400dc60 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 shutdown_connections 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe04c1080a0 0x7fe04c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe04c102e70 0x7fe04c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe0380b2880 0x7fe0380b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 --2- 172.21.15.50:0/3068527765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe04c107750 0x7fe04c19ad40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:13.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.964+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 >> 172.21.15.50:0/3068527765 conn(0x7fe04c079ae0 msgr2=0x7fe04c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:13.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.965+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 shutdown_connections 2024-05-17T04:31:13.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:13.965+0000 7fe05269f700 1 -- 172.21.15.50:0/3068527765 wait complete. 2024-05-17T04:31:13.967 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 8 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: Deploying daemon mds.h on smithi105 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:boot 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: fsmap 7 up:standby 2024-05-17T04:31:14.067 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:13 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: Deploying daemon mds.h on smithi105 2024-05-17T04:31:14.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:boot 2024-05-17T04:31:14.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: fsmap 7 up:standby 2024-05-17T04:31:14.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:13 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:31:14.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:14 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3068527765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:15.048 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:14 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3068527765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:15.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:14 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3068527765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:15.314 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 podman[64620]: 2024-05-17 04:31:15.170878064 +0000 UTC m=+0.017235894 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:15.315 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 podman[64620]: 2024-05-17 04:31:15.315167024 +0000 UTC m=+0.161524841 container create fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:15 smithi105 ceph-mon[34795]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:15 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:15 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 podman[64620]: 2024-05-17 04:31:15.569003915 +0000 UTC m=+0.415361742 container init fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 podman[64620]: 2024-05-17 04:31:15.571980041 +0000 UTC m=+0.418337858 container start fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 ceph-mds[64639]: -- 172.21.15.105:0/4073925949 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d93463a40 con 0x562d934a9600 2024-05-17T04:31:16.065 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: starting mds.h at 2024-05-17T04:31:16.066 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 bash[64620]: fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 2024-05-17T04:31:16.066 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:31:15 smithi105 systemd[1]: Started Ceph mds.h for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:15 smithi110 ceph-mon[35497]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:15 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:15 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:15 smithi050 ceph-mon[31739]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:15 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:15 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.598 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:16.863 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:16.985 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6836/1162023031,v1:172.21.15.105:6837/1162023031] up:boot 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: fsmap 8 up:standby 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:16.986 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:16 smithi050 ceph-mon[31739]: Deploying daemon mds.i on smithi110 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6836/1162023031,v1:172.21.15.105:6837/1162023031] up:boot 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: fsmap 8 up:standby 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:17.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:17.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:17.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:16 smithi105 ceph-mon[34795]: Deploying daemon mds.i on smithi110 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6836/1162023031,v1:172.21.15.105:6837/1162023031] up:boot 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: fsmap 8 up:standby 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:17.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:17.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:17.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:16 smithi110 ceph-mon[35497]: Deploying daemon mds.i on smithi110 2024-05-17T04:31:18.021 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:17 smithi050 ceph-mon[31739]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:18.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:17 smithi105 ceph-mon[34795]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:18.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:17 smithi110 ceph-mon[35497]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:18.067 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:17 smithi110 systemd[1]: Starting Ceph mds.i for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:18.466 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 podman[66635]: 2024-05-17 04:31:18.097407536 +0000 UTC m=+0.017000554 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:18.466 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 podman[66635]: 2024-05-17 04:31:18.213490591 +0000 UTC m=+0.133083643 container create 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 -- 172.21.15.50:0/1335592669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 msgr2=0x7fc1a410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/1335592669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a410dd10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b0a0 tx=0x7fc19400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 -- 172.21.15.50:0/1335592669 shutdown_connections 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/1335592669 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 0x7fc1a4110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/1335592669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 0x7fc1a4105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/1335592669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a410dd10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 -- 172.21.15.50:0/1335592669 >> 172.21.15.50:0/1335592669 conn(0x7fc1a4079ae0 msgr2=0x7fc1a4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:18.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 -- 172.21.15.50:0/1335592669 shutdown_connections 2024-05-17T04:31:18.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.589+0000 7fc1aacc4700 1 -- 172.21.15.50:0/1335592669 wait complete. 2024-05-17T04:31:18.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 Processor -- start 2024-05-17T04:31:18.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 -- start start 2024-05-17T04:31:18.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 0x7fc1a419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:18.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:18.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 0x7fc1a419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:18.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4113cc0 con 0x7fc1a41057c0 2024-05-17T04:31:18.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4113b40 con 0x7fc1a4104e70 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1aacc4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4113e40 con 0x7fc1a410e2a0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1a3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1a3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35038/0 (socket says 172.21.15.50:35038) 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1a3fff700 1 -- 172.21.15.50:0/864565874 learned_addr learned my addr 172.21.15.50:0/864565874 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1a8a60700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 0x7fc1a419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.590+0000 7fc1a9261700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 0x7fc1a419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 -- 172.21.15.50:0/864565874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 msgr2=0x7fc1a419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 0x7fc1a419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 -- 172.21.15.50:0/864565874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 msgr2=0x7fc1a419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 0x7fc1a419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 -- 172.21.15.50:0/864565874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a419f5f0 con 0x7fc1a41057c0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a3fff700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b070 tx=0x7fc194004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194021070 con 0x7fc1a41057c0 2024-05-17T04:31:18.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc194007b80 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a419f880 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004740 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.591+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a419fd60 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.592+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a4105280 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.594+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc194003a00 con 0x7fc1a41057c0 2024-05-17T04:31:18.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.595+0000 7fc1a1ffb700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 0x7fc1900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:18.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.595+0000 7fc1a8a60700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 0x7fc1900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:18.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.595+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc19400f020 con 0x7fc1a41057c0 2024-05-17T04:31:18.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.595+0000 7fc1a8a60700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 0x7fc1900b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005950 tx=0x7fc18c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:18.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.598+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc19402a020 con 0x7fc1a41057c0 2024-05-17T04:31:18.817 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 podman[66635]: 2024-05-17 04:31:18.466759549 +0000 UTC m=+0.386352556 container init 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T04:31:18.818 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 podman[66635]: 2024-05-17 04:31:18.469805945 +0000 UTC m=+0.389398959 container start 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-17T04:31:18.818 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 ceph-mds[66654]: -- 172.21.15.110:0/3812111779 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56119b4eda40 con 0x56119b533a80 2024-05-17T04:31:18.818 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: starting mds.i at 2024-05-17T04:31:18.818 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 bash[66635]: 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 2024-05-17T04:31:18.818 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:31:18 smithi110 systemd[1]: Started Ceph mds.i for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:18.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.876+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a419ba80 con 0x7fc1a41057c0 2024-05-17T04:31:18.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.877+0000 7fc1a1ffb700 1 -- 172.21.15.50:0/864565874 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fc1940ab8e0 con 0x7fc1a41057c0 2024-05-17T04:31:18.877 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:18.877 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-17T04:31:18.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 msgr2=0x7fc1900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 0x7fc1900b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005950 tx=0x7fc18c00a300 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 msgr2=0x7fc1a419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b070 tx=0x7fc194004d50 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 shutdown_connections 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a410e2a0 0x7fc1a419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a4104e70 0x7fc1a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b2880 0x7fc1900b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 --2- 172.21.15.50:0/864565874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a41057c0 0x7fc1a419acc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 >> 172.21.15.50:0/864565874 conn(0x7fc1a4079ae0 msgr2=0x7fc1a40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 shutdown_connections 2024-05-17T04:31:18.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:18.879+0000 7fc1aacc4700 1 -- 172.21.15.50:0/864565874 wait complete. 2024-05-17T04:31:18.881 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 10 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:19.811 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: Deploying daemon mds.g on smithi050 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:boot 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: fsmap 9 up:standby 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:31:19.812 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:19 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/864565874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: Deploying daemon mds.g on smithi050 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:boot 2024-05-17T04:31:20.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: fsmap 9 up:standby 2024-05-17T04:31:20.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:31:20.066 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:19 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/864565874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: Deploying daemon mds.g on smithi050 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:20.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:boot 2024-05-17T04:31:20.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: fsmap 9 up:standby 2024-05-17T04:31:20.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:31:20.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:19 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/864565874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:21.529 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 systemd[1]: Starting Ceph mds.g for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:21.562 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:21.782 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:21 smithi050 ceph-mon[31739]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:21.783 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 podman[70532]: 2024-05-17 04:31:21.432364799 +0000 UTC m=+0.019019550 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:21.783 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 podman[70532]: 2024-05-17 04:31:21.546544863 +0000 UTC m=+0.133199593 container create bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-17T04:31:21.783 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 podman[70532]: 2024-05-17 04:31:21.782965217 +0000 UTC m=+0.369619946 container init bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-17T04:31:21.939 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:22.063 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 podman[70532]: 2024-05-17 04:31:21.786337312 +0000 UTC m=+0.372992063 container start bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-17T04:31:22.064 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 ceph-mds[70581]: -- 172.21.15.50:0/1568289996 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55556c6f9a40 con 0x55556c73fa80 2024-05-17T04:31:22.064 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g[70565]: starting mds.g at 2024-05-17T04:31:22.064 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 bash[70532]: bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c 2024-05-17T04:31:22.064 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:31:21 smithi050 systemd[1]: Started Ceph mds.g for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:22.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:21 smithi105 ceph-mon[34795]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:22.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:21 smithi110 ceph-mon[35497]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:23.169 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:23.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: Deploying daemon mds.a on smithi050 2024-05-17T04:31:23.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.50:6838/3126760693,v1:172.21.15.50:6839/3126760693] up:boot 2024-05-17T04:31:23.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: fsmap 10 up:standby 2024-05-17T04:31:23.170 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:22 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T04:31:23.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: Deploying daemon mds.a on smithi050 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.50:6838/3126760693,v1:172.21.15.50:6839/3126760693] up:boot 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: fsmap 10 up:standby 2024-05-17T04:31:23.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:22 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: Deploying daemon mds.a on smithi050 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.50:6838/3126760693,v1:172.21.15.50:6839/3126760693] up:boot 2024-05-17T04:31:23.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: fsmap 10 up:standby 2024-05-17T04:31:23.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:22 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-17T04:31:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:23 smithi105 ceph-mon[34795]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:23 smithi110 ceph-mon[35497]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:23 smithi050 ceph-mon[31739]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1485261488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c84107750 msgr2=0x7f8c84107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1485261488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c84107750 0x7f8c84107b60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009990 tx=0x7f8c78009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1485261488 shutdown_connections 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1485261488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c841080a0 0x7f8c8410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1485261488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 0x7f8c84103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1485261488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c84107750 0x7f8c84107b60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1485261488 >> 172.21.15.50:0/1485261488 conn(0x7f8c84079ae0 msgr2=0x7f8c84079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1485261488 shutdown_connections 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.227+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1485261488 wait complete. 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 Processor -- start 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 -- start start 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 0x7f8c841a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c84107750 0x7f8c841a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112b60 con 0x7f8c841080a0 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841129e0 con 0x7f8c84107750 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8aec2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112ce0 con 0x7f8c84102e70 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48024/0 (socket says 172.21.15.50:48024) 2024-05-17T04:31:25.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 -- 172.21.15.50:0/1969883558 learned_addr learned my addr 172.21.15.50:0/1969883558 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c88c5e700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 0x7f8c841a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c83fff700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c84107750 0x7f8c841a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 -- 172.21.15.50:0/1969883558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 msgr2=0x7f8c841a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 0x7f8c841a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 -- 172.21.15.50:0/1969883558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c84107750 msgr2=0x7f8c841a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c84107750 0x7f8c841a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.228+0000 7f8c8945f700 1 -- 172.21.15.50:0/1969883558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8419ce50 con 0x7f8c841080a0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c8945f700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007ee0 tx=0x7f8c7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004bb0 con 0x7f8c841080a0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c7c004d50 con 0x7f8c841080a0 2024-05-17T04:31:25.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8419d140 con 0x7f8c841080a0 2024-05-17T04:31:25.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0058b0 con 0x7f8c841080a0 2024-05-17T04:31:25.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.229+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a9340 con 0x7f8c841080a0 2024-05-17T04:31:25.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.230+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c841042d0 con 0x7f8c841080a0 2024-05-17T04:31:25.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.232+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8c7c01a020 con 0x7f8c841080a0 2024-05-17T04:31:25.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.232+0000 7f8c81ffb700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 0x7f8c6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:25.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.232+0000 7f8c88c5e700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 0x7f8c6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:25.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.232+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f8c7c0e1340 con 0x7f8c841080a0 2024-05-17T04:31:25.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.233+0000 7f8c88c5e700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 0x7f8c6c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005950 tx=0x7f8c7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:25.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.237+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c7c010070 con 0x7f8c841080a0 2024-05-17T04:31:25.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.531+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c84103280 con 0x7f8c841080a0 2024-05-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.531+0000 7f8c81ffb700 1 -- 172.21.15.50:0/1969883558 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f8c7c0a9500 con 0x7f8c841080a0 2024-05-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:25.532 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 msgr2=0x7f8c6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 0x7f8c6c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005950 tx=0x7f8c7400b410 comp rx=0 tx=0).stop 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 msgr2=0x7f8c8419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c007ee0 tx=0x7f8c7c00d3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 shutdown_connections 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c6c0b2900 0x7f8c6c0b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c841080a0 0x7f8c8419c720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c84102e70 0x7f8c841a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 --2- 172.21.15.50:0/1969883558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c84107750 0x7f8c841a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 >> 172.21.15.50:0/1969883558 conn(0x7f8c84079ae0 msgr2=0x7f8c8410e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 shutdown_connections 2024-05-17T04:31:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:25.534+0000 7f8c8aec2700 1 -- 172.21.15.50:0/1969883558 wait complete. 2024-05-17T04:31:25.535 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 11 2024-05-17T04:31:26.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:25 smithi050 ceph-mon[31739]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:26.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:25 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1969883558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:25 smithi105 ceph-mon[34795]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:26.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:25 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1969883558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:25 smithi110 ceph-mon[35497]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:25 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1969883558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:27.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.50:6840/3896696320,v1:172.21.15.50:6841/3896696320] up:boot 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: fsmap 11 up:standby 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:26 smithi105 ceph-mon[34795]: Deploying daemon mds.b on smithi105 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.50:6840/3896696320,v1:172.21.15.50:6841/3896696320] up:boot 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: fsmap 11 up:standby 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:27.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:27.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:27.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:26 smithi110 ceph-mon[35497]: Deploying daemon mds.b on smithi105 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.50:6840/3896696320,v1:172.21.15.50:6841/3896696320] up:boot 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: fsmap 11 up:standby 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:27.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-17T04:31:27.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-17T04:31:27.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:27.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:26 smithi050 ceph-mon[31739]: Deploying daemon mds.b on smithi105 2024-05-17T04:31:28.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:28 smithi105 ceph-mon[34795]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:28.341 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 systemd[1]: Starting Ceph mds.b for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:31:28.342 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:28 smithi050 ceph-mon[31739]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:28.342 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:28 smithi110 ceph-mon[35497]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:28.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:31:28.668 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 podman[64926]: 2024-05-17 04:31:28.300748713 +0000 UTC m=+0.017651750 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:31:28.668 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 podman[64926]: 2024-05-17 04:31:28.404337393 +0000 UTC m=+0.121240414 container create 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T04:31:28.668 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 podman[64926]: 2024-05-17 04:31:28.631887194 +0000 UTC m=+0.348790224 container init 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-17T04:31:28.668 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 podman[64926]: 2024-05-17 04:31:28.634565623 +0000 UTC m=+0.351468656 container start 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-17T04:31:28.668 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 ceph-mds[64946]: -- 172.21.15.105:0/506156628 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56012acefa40 con 0x56012ad35180 2024-05-17T04:31:28.732 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:29.064 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: starting mds.b at 2024-05-17T04:31:29.064 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 bash[64926]: 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 2024-05-17T04:31:29.065 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:28 smithi105 systemd[1]: Started Ceph mds.b for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:31:30.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:30.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.047 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:30.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:boot 2024-05-17T04:31:30.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: fsmap 12 up:standby 2024-05-17T04:31:30.048 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:boot 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: fsmap 12 up:standby 2024-05-17T04:31:30.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:boot 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: fsmap 12 up:standby 2024-05-17T04:31:30.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/969961742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0105710 msgr2=0x7f5cb010dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/969961742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0105710 0x7f5cb010dc60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009990 tx=0x7f5c9c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/969961742 shutdown_connections 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/969961742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 0x7f5cb0110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/969961742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0104dc0 0x7f5cb01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/969961742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0105710 0x7f5cb010dc60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.433+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/969961742 >> 172.21.15.50:0/969961742 conn(0x7f5cb0079bd0 msgr2=0x7f5cb0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.434+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/969961742 shutdown_connections 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.434+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/969961742 wait complete. 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.434+0000 7f5cb6a8f700 1 Processor -- start 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.434+0000 7f5cb6a8f700 1 -- start start 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 0x7f5cb019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 0x7f5cb019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0113c50 con 0x7f5cb0104dc0 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0113ad0 con 0x7f5cb0105710 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb6a8f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0113dd0 con 0x7f5cb010e220 2024-05-17T04:31:30.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5caffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 0x7f5cb019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58954/0 (socket says 172.21.15.50:58954) 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 -- 172.21.15.50:0/2821685557 learned_addr learned my addr 172.21.15.50:0/2821685557 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb502c700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 0x7f5cb019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 -- 172.21.15.50:0/2821685557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 msgr2=0x7f5cb019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 0x7f5cb019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 -- 172.21.15.50:0/2821685557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 msgr2=0x7f5cb019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 0x7f5cb019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 -- 172.21.15.50:0/2821685557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb019f580 con 0x7f5cb0104dc0 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5caffff700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 0x7f5cb019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.435+0000 7f5cb482b700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9a0 tx=0x7f5ca400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.436+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400ca10 con 0x7f5cb0104dc0 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.436+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb019f870 con 0x7f5cb0104dc0 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.436+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ca400cbb0 con 0x7f5cb0104dc0 2024-05-17T04:31:30.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.436+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018680 con 0x7f5cb0104dc0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.436+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01a7190 con 0x7f5cb0104dc0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.437+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb01051d0 con 0x7f5cb0104dc0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.439+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5ca4010430 con 0x7f5cb0104dc0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.439+0000 7f5cadffb700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 0x7f5ca00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.439+0000 7f5caffff700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 0x7f5ca00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.439+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5ca40e22c0 con 0x7f5cb0104dc0 2024-05-17T04:31:30.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.440+0000 7f5caffff700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 0x7f5ca00b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5cb010fe80 tx=0x7f5c9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:30.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.443+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ca40aa480 con 0x7f5cb0104dc0 2024-05-17T04:31:30.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.740+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb002ea80 con 0x7f5cb0104dc0 2024-05-17T04:31:30.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.741+0000 7f5cadffb700 1 -- 172.21.15.50:0/2821685557 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f5ca40a9d80 con 0x7f5cb0104dc0 2024-05-17T04:31:30.742 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:30.743 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.744+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 msgr2=0x7f5ca00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.744+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 0x7f5ca00b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5cb010fe80 tx=0x7f5c9c005fb0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.744+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 msgr2=0x7f5cb019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.744+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9a0 tx=0x7f5ca400ecb0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 shutdown_connections 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5cb010e220 0x7f5cb019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5ca00b2880 0x7f5ca00b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cb0104dc0 0x7f5cb019a6b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 --2- 172.21.15.50:0/2821685557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cb0105710 0x7f5cb019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 >> 172.21.15.50:0/2821685557 conn(0x7f5cb0079bd0 msgr2=0x7f5cb00ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 shutdown_connections 2024-05-17T04:31:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:30.745+0000 7f5cb6a8f700 1 -- 172.21.15.50:0/2821685557 wait complete. 2024-05-17T04:31:30.747 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 13 2024-05-17T04:31:30.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:30 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2821685557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:30.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:30.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:31.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:30 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2821685557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:31.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:30 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2821685557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:31:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:31.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:31 smithi050 ceph-mon[31739]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:32.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.028 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.058 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-17T04:31:32.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:31 smithi105 ceph-mon[34795]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:32.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:31 smithi110 ceph-mon[35497]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:32.328 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:34.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1386573829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38102590 msgr2=0x7f1d381029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1386573829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38102590 0x7f1d381029a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009990 tx=0x7f1d28009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1386573829 shutdown_connections 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1386573829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38107810 0x7f1d3810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1386573829 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38102590 0x7f1d381029a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1386573829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38106ec0 0x7f1d381072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.110+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1386573829 >> 172.21.15.50:0/1386573829 conn(0x7f1d38078040 msgr2=0x7f1d380784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:34.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.111+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1386573829 shutdown_connections 2024-05-17T04:31:34.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.111+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1386573829 wait complete. 2024-05-17T04:31:34.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 Processor -- start 2024-05-17T04:31:34.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 -- start start 2024-05-17T04:31:34.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 0x7f1d3819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:34.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38106ec0 0x7f1d3819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:34.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38113d20 con 0x7f1d38107810 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38113ba0 con 0x7f1d38106ec0 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d3dd3d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38113ea0 con 0x7f1d38102590 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d377fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 0x7f1d3819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:58976/0 (socket says 172.21.15.50:58976) 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d37fff700 1 -- 172.21.15.50:0/1115636343 learned_addr learned my addr 172.21.15.50:0/1115636343 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.112+0000 7f1d2effd700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38106ec0 0x7f1d3819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 -- 172.21.15.50:0/1115636343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 msgr2=0x7f1d3819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 0x7f1d3819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 -- 172.21.15.50:0/1115636343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38106ec0 msgr2=0x7f1d3819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:34.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38106ec0 0x7f1d3819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 -- 172.21.15.50:0/1115636343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819f5b0 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d377fe700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 0x7f1d3819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d37fff700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ef30 tx=0x7f1d3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300092e0 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819f840 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d30004bb0 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004020 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.113+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3819fd30 con 0x7f1d38107810 2024-05-17T04:31:34.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.115+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d381029a0 con 0x7f1d38107810 2024-05-17T04:31:34.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.117+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1d3001b030 con 0x7f1d38107810 2024-05-17T04:31:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.118+0000 7f1d357fa700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 0x7f1d240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.118+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1d300e2290 con 0x7f1d38107810 2024-05-17T04:31:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.118+0000 7f1d377fe700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 0x7f1d240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:34.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.119+0000 7f1d377fe700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 0x7f1d240b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800b2e0 tx=0x7f1d28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:34.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.120+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d30022020 con 0x7f1d38107810 2024-05-17T04:31:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:34.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:34.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:34.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.338+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1d3819ba40 con 0x7f1d38107810 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.339+0000 7f1d357fa700 1 -- 172.21.15.50:0/1115636343 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f1d30010040 con 0x7f1d38107810 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout:[ 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout: "poolnum": 1, 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout: "poolname": ".mgr" 2024-05-17T04:31:34.340 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:31:34.341 INFO:teuthology.orchestra.run.smithi050.stdout:] 2024-05-17T04:31:34.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 msgr2=0x7f1d240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 0x7f1d240b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800b2e0 tx=0x7f1d28005fb0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 msgr2=0x7f1d3819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ef30 tx=0x7f1d3000c3a0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 shutdown_connections 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d240b2880 0x7f1d240b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d38107810 0x7f1d3819ef10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d38102590 0x7f1d3819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 --2- 172.21.15.50:0/1115636343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d38106ec0 0x7f1d3819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 >> 172.21.15.50:0/1115636343 conn(0x7f1d38078040 msgr2=0x7f1d38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.341+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 shutdown_connections 2024-05-17T04:31:34.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:34.342+0000 7f1d3dd3d700 1 -- 172.21.15.50:0/1115636343 wait complete. 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:31:35.030 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T04:31:35.030 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T04:31:35.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:35 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1115636343' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T04:31:35.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:35 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1115636343' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T04:31:35.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:35 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1115636343' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-17T04:31:35.867 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:36.257 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:36 smithi050 ceph-mon[31739]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:36.257 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.257 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.257 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.257 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:36 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:36 smithi105 ceph-mon[34795]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:36 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:36 smithi110 ceph-mon[35497]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:36 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:37.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 -- 172.21.15.50:0/3926442103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c108080 msgr2=0x7f633c10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 --2- 172.21.15.50:0/3926442103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c108080 0x7f633c10f9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f633400b0a0 tx=0x7f633400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 -- 172.21.15.50:0/3926442103 shutdown_connections 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 --2- 172.21.15.50:0/3926442103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c108080 0x7f633c10f9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 --2- 172.21.15.50:0/3926442103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c102e00 0x7f633c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 --2- 172.21.15.50:0/3926442103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c107730 0x7f633c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 -- 172.21.15.50:0/3926442103 >> 172.21.15.50:0/3926442103 conn(0x7f633c079bd0 msgr2=0x7f633c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 -- 172.21.15.50:0/3926442103 shutdown_connections 2024-05-17T04:31:37.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.655+0000 7f63411a3700 1 -- 172.21.15.50:0/3926442103 wait complete. 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.656+0000 7f63411a3700 1 Processor -- start 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.656+0000 7f63411a3700 1 -- start start 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c108080 0x7f633c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f633c112b70 con 0x7f633c102e00 2024-05-17T04:31:37.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1129f0 con 0x7f633c108080 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f63411a3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f633c112cf0 con 0x7f633c107730 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633ad9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633ad9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59002/0 (socket says 172.21.15.50:59002) 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:45444/0 (socket says 172.21.15.50:45444) 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633ad9d700 1 -- 172.21.15.50:0/2208089051 learned_addr learned my addr 172.21.15.50:0/2208089051 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633b59e700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c108080 0x7f633c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 -- 172.21.15.50:0/2208089051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c108080 msgr2=0x7f633c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c108080 0x7f633c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 -- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 msgr2=0x7f633c1a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:37.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633259c700 1 -- 172.21.15.50:0/2208089051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c19acc0 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.657+0000 7f633ad9d700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f633259c700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f632400d720 tx=0x7f632400da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324014030 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63240044b0 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240052a0 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c19af50 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.658+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c19b440 con 0x7f633c107730 2024-05-17T04:31:37.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.659+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c103210 con 0x7f633c107730 2024-05-17T04:31:37.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.661+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6324004020 con 0x7f633c107730 2024-05-17T04:31:37.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.662+0000 7f6338d99700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 0x7f63280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:37.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.662+0000 7f633ad9d700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 0x7f63280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:37.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.662+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f63240e1960 con 0x7f633c107730 2024-05-17T04:31:37.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.663+0000 7f633ad9d700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 0x7f63280b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f633c19bac0 tx=0x7f632c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:37.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.665+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6324004750 con 0x7f633c107730 2024-05-17T04:31:37.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:37.899+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f633c19bb60 con 0x7f633c107730 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2208089051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:38 smithi105 ceph-mon[34795]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2208089051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:38 smithi110 ceph-mon[35497]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:31:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:31:38.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:31:38.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2208089051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:38 smithi050 ceph-mon[31739]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-17T04:31:38.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.880+0000 7f6338d99700 1 -- 172.21.15.50:0/2208089051 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f63240a9a80 con 0x7f633c107730 2024-05-17T04:31:38.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 msgr2=0x7f63280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 0x7f63280b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f633c19bac0 tx=0x7f632c005f90 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 msgr2=0x7f633c1a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f632400d720 tx=0x7f632400da30 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 shutdown_connections 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633c108080 0x7f633c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f63280b2900 0x7f63280b4e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f633c102e00 0x7f633c1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 --2- 172.21.15.50:0/2208089051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f633c107730 0x7f633c1a09b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 >> 172.21.15.50:0/2208089051 conn(0x7f633c079bd0 msgr2=0x7f633c06ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 shutdown_connections 2024-05-17T04:31:38.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:38.885+0000 7f63411a3700 1 -- 172.21.15.50:0/2208089051 wait complete. 2024-05-17T04:31:38.888 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' created 2024-05-17T04:31:39.565 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-17T04:31:39.823 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:40.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:39 smithi050 ceph-mon[31739]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:40.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:39 smithi050 ceph-mon[31739]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T04:31:40.167 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:39 smithi050 ceph-mon[31739]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T04:31:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:39 smithi105 ceph-mon[34795]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:39 smithi105 ceph-mon[34795]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T04:31:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:39 smithi105 ceph-mon[34795]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T04:31:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:39 smithi110 ceph-mon[35497]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:39 smithi110 ceph-mon[35497]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-17T04:31:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:39 smithi110 ceph-mon[35497]: osdmap e72: 12 total, 12 up, 12 in 2024-05-17T04:31:41.238 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:40 smithi050 ceph-mon[31739]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T04:31:41.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:40 smithi105 ceph-mon[34795]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T04:31:41.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:40 smithi110 ceph-mon[35497]: osdmap e73: 12 total, 12 up, 12 in 2024-05-17T04:31:41.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1222573857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc102480 msgr2=0x7fa9bc102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1222573857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc102480 0x7fa9bc102890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009990 tx=0x7fa9ac009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1222573857 shutdown_connections 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1222573857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1222573857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc102480 0x7fa9bc102890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1222573857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 0x7fa9bc108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1222573857 >> 172.21.15.50:0/1222573857 conn(0x7fa9bc0fe1a0 msgr2=0x7fa9bc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1222573857 shutdown_connections 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1222573857 wait complete. 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.627+0000 7fa9c13cb700 1 Processor -- start 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 -- start start 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc102480 0x7fa9bc19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:41.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 0x7fa9bc19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc113d00 con 0x7fa9bc1091f0 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc113b80 con 0x7fa9bc1088a0 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9c13cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc113e80 con 0x7fa9bc102480 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9bb7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9bb7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47428/0 (socket says 172.21.15.50:47428) 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9bb7fe700 1 -- 172.21.15.50:0/1271476861 learned_addr learned my addr 172.21.15.50:0/1271476861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9baffd700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc102480 0x7fa9bc19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.628+0000 7fa9ba7fc700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 0x7fa9bc19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 -- 172.21.15.50:0/1271476861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc102480 msgr2=0x7fa9bc19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc102480 0x7fa9bc19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 -- 172.21.15.50:0/1271476861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 msgr2=0x7fa9bc19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:41.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 0x7fa9bc19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 -- 172.21.15.50:0/1271476861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc19f670 con 0x7fa9bc1091f0 2024-05-17T04:31:41.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9bb7fe700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007ee0 tx=0x7fa9b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:41.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4004bb0 con 0x7fa9bc1091f0 2024-05-17T04:31:41.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc19f960 con 0x7fa9bc1091f0 2024-05-17T04:31:41.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b4004d50 con 0x7fa9bc1091f0 2024-05-17T04:31:41.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40058b0 con 0x7fa9bc1091f0 2024-05-17T04:31:41.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.629+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1a7280 con 0x7fa9bc1091f0 2024-05-17T04:31:41.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.630+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc102890 con 0x7fa9bc1091f0 2024-05-17T04:31:41.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.632+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa9b401a020 con 0x7fa9bc1091f0 2024-05-17T04:31:41.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.633+0000 7fa9a3fff700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 0x7fa9a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:41.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.633+0000 7fa9baffd700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 0x7fa9a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:41.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.633+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fa9b40e1c30 con 0x7fa9bc1091f0 2024-05-17T04:31:41.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.633+0000 7fa9baffd700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 0x7fa9a40b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b5c0 tx=0x7fa9ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:41.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.636+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9b4010070 con 0x7fa9bc1091f0 2024-05-17T04:31:41.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.853+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa9bc19bb00 con 0x7fa9bc1091f0 2024-05-17T04:31:41.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.900+0000 7fa9a3fff700 1 -- 172.21.15.50:0/1271476861 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fa9b40a9be0 con 0x7fa9bc1091f0 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 msgr2=0x7fa9a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 0x7fa9a40b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b5c0 tx=0x7fa9ac005960 comp rx=0 tx=0).stop 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 msgr2=0x7fa9bc19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007ee0 tx=0x7fa9b400d3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 shutdown_connections 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa9a40b2880 0x7fa9a40b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9bc1091f0 0x7fa9bc19efd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9bc102480 0x7fa9bc19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 --2- 172.21.15.50:0/1271476861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9bc1088a0 0x7fa9bc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:41.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.902+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 >> 172.21.15.50:0/1271476861 conn(0x7fa9bc0fe1a0 msgr2=0x7fa9bc101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:41.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.903+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 shutdown_connections 2024-05-17T04:31:41.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:41.903+0000 7fa9c13cb700 1 -- 172.21.15.50:0/1271476861 wait complete. 2024-05-17T04:31:41.912 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' created 2024-05-17T04:31:42.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:41 smithi050 ceph-mon[31739]: pgmap v284: 33 pgs: 4 active+clean, 12 creating+peering, 17 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:42.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:41 smithi050 ceph-mon[31739]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:42.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:41 smithi050 ceph-mon[31739]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T04:31:42.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:41 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T04:31:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:41 smithi105 ceph-mon[34795]: pgmap v284: 33 pgs: 4 active+clean, 12 creating+peering, 17 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:41 smithi105 ceph-mon[34795]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:42.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:41 smithi105 ceph-mon[34795]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T04:31:42.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:41 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T04:31:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:41 smithi110 ceph-mon[35497]: pgmap v284: 33 pgs: 4 active+clean, 12 creating+peering, 17 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:41 smithi110 ceph-mon[35497]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:41 smithi110 ceph-mon[35497]: osdmap e74: 12 total, 12 up, 12 in 2024-05-17T04:31:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:41 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-17T04:31:42.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T04:31:43.028 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:43.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:42 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T04:31:43.066 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:42 smithi050 ceph-mon[31739]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T04:31:43.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:42 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T04:31:43.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:42 smithi105 ceph-mon[34795]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T04:31:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:42 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1271476861' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-17T04:31:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:42 smithi110 ceph-mon[35497]: osdmap e75: 12 total, 12 up, 12 in 2024-05-17T04:31:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:43 smithi105 ceph-mon[34795]: pgmap v287: 97 pgs: 16 active+clean, 20 creating+peering, 61 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:44.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:43 smithi105 ceph-mon[34795]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T04:31:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:43 smithi110 ceph-mon[35497]: pgmap v287: 97 pgs: 16 active+clean, 20 creating+peering, 61 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:43 smithi110 ceph-mon[35497]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T04:31:44.319 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:43 smithi050 ceph-mon[31739]: pgmap v287: 97 pgs: 16 active+clean, 20 creating+peering, 61 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:44.319 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:43 smithi050 ceph-mon[31739]: osdmap e76: 12 total, 12 up, 12 in 2024-05-17T04:31:44.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 -- 172.21.15.50:0/3659522382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 msgr2=0x7f6690110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 --2- 172.21.15.50:0/3659522382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f6690110af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6688009990 tx=0x7f6688009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 -- 172.21.15.50:0/3659522382 shutdown_connections 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 --2- 172.21.15.50:0/3659522382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f6690110af0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 --2- 172.21.15.50:0/3659522382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690103f10 0x7f6690104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 --2- 172.21.15.50:0/3659522382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690108840 0x7f6690108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 -- 172.21.15.50:0/3659522382 >> 172.21.15.50:0/3659522382 conn(0x7f66900ff9b0 msgr2=0x7f6690101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 -- 172.21.15.50:0/3659522382 shutdown_connections 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.876+0000 7f6696e1d700 1 -- 172.21.15.50:0/3659522382 wait complete. 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 Processor -- start 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 -- start start 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690103f10 0x7f66901a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:44.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690108840 0x7f66901a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113cf0 con 0x7f6690109190 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113b70 con 0x7f6690108840 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.877+0000 7f6696e1d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6690113e70 con 0x7f6690103f10 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47434/0 (socket says 172.21.15.50:47434) 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 -- 172.21.15.50:0/519279177 learned_addr learned my addr 172.21.15.50:0/519279177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f6694bb9700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690103f10 0x7f66901a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f668ffff700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690108840 0x7f66901a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 -- 172.21.15.50:0/519279177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690103f10 msgr2=0x7f66901a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690103f10 0x7f66901a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 -- 172.21.15.50:0/519279177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690108840 msgr2=0x7f66901a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690108840 0x7f66901a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:44.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 -- 172.21.15.50:0/519279177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66901acf90 con 0x7f6690109190 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f66953ba700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6688005910 tx=0x7f66880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004830 con 0x7f6690109190 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66901ad160 con 0x7f6690109190 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6688005020 con 0x7f6690109190 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.878+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800f950 con 0x7f6690109190 2024-05-17T04:31:44.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.879+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66901ad610 con 0x7f6690109190 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.880+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6690104320 con 0x7f6690109190 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.881+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f66880054f0 con 0x7f6690109190 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.882+0000 7f668dffb700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 0x7f667c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.882+0000 7f6694bb9700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 0x7f667c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.882+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f66880e3400 con 0x7f6690109190 2024-05-17T04:31:44.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.883+0000 7f6694bb9700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 0x7f667c0b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66800060b0 tx=0x7f668000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:44.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:44.885+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6688013070 con 0x7f6690109190 2024-05-17T04:31:45.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.148+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6690110660 con 0x7f6690109190 2024-05-17T04:31:45.149 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:44 smithi050 ceph-mon[31739]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T04:31:45.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:44 smithi105 ceph-mon[34795]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T04:31:45.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:44 smithi110 ceph-mon[35497]: osdmap e77: 12 total, 12 up, 12 in 2024-05-17T04:31:45.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.922+0000 7f668dffb700 1 -- 172.21.15.50:0/519279177 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f66880ab2c0 con 0x7f6690109190 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 msgr2=0x7f667c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 0x7f667c0b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66800060b0 tx=0x7f668000c040 comp rx=0 tx=0).stop 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 msgr2=0x7f66901aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6688005910 tx=0x7f66880047b0 comp rx=0 tx=0).stop 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 shutdown_connections 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f667c0b2900 0x7f667c0b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:45.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6690109190 0x7f66901aca50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6690103f10 0x7f66901a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 --2- 172.21.15.50:0/519279177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6690108840 0x7f66901a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 >> 172.21.15.50:0/519279177 conn(0x7f66900ff9b0 msgr2=0x7f669010a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 shutdown_connections 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:45.926+0000 7f6696e1d700 1 -- 172.21.15.50:0/519279177 wait complete. 2024-05-17T04:31:45.927 INFO:teuthology.orchestra.run.smithi050.stderr:new fs with metadata pool 2 and data pool 3 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:45 smithi050 ceph-mon[31739]: pgmap v290: 97 pgs: 52 active+clean, 16 creating+peering, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:45 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:45 smithi050 ceph-mon[31739]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:45 smithi105 ceph-mon[34795]: pgmap v290: 97 pgs: 52 active+clean, 16 creating+peering, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:45 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T04:31:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:45 smithi105 ceph-mon[34795]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T04:31:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:45 smithi110 ceph-mon[35497]: pgmap v290: 97 pgs: 52 active+clean, 16 creating+peering, 29 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:45 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-17T04:31:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:45 smithi110 ceph-mon[35497]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-17T04:31:46.757 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-17T04:31:46.757 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-17T04:31:47.019 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:46 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T04:31:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:46 smithi050 ceph-mon[31739]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T04:31:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:46 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T04:31:47.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:46 smithi050 ceph-mon[31739]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T04:31:47.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:46 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T04:31:47.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:46 smithi105 ceph-mon[34795]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T04:31:47.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:46 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T04:31:47.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:46 smithi105 ceph-mon[34795]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T04:31:47.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:46 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/519279177' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-17T04:31:47.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:46 smithi110 ceph-mon[35497]: osdmap e78: 12 total, 12 up, 12 in 2024-05-17T04:31:47.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:46 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-17T04:31:47.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:46 smithi110 ceph-mon[35497]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-17T04:31:48.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: pgmap v292: 97 pgs: 74 active+clean, 16 creating+peering, 7 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T04:31:48.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T04:31:48.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: Cluster is now healthy 2024-05-17T04:31:48.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:active 2024-05-17T04:31:48.281 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:47 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: pgmap v292: 97 pgs: 74 active+clean, 16 creating+peering, 7 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T04:31:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T04:31:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: Cluster is now healthy 2024-05-17T04:31:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:active 2024-05-17T04:31:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:47 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: pgmap v292: 97 pgs: 74 active+clean, 16 creating+peering, 7 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-17T04:31:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-17T04:31:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: Cluster is now healthy 2024-05-17T04:31:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] up:active 2024-05-17T04:31:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:47 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 -- 172.21.15.50:0/3372963612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0109190 msgr2=0x7f7fa0110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/3372963612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0109190 0x7f7fa0110af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800b0a0 tx=0x7f7f9800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 -- 172.21.15.50:0/3372963612 shutdown_connections 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/3372963612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0109190 0x7f7fa0110af0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/3372963612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0103f10 0x7f7fa0104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/3372963612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 -- 172.21.15.50:0/3372963612 >> 172.21.15.50:0/3372963612 conn(0x7f7fa00ff9b0 msgr2=0x7f7fa0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 -- 172.21.15.50:0/3372963612 shutdown_connections 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.868+0000 7f7fa63aa700 1 -- 172.21.15.50:0/3372963612 wait complete. 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 Processor -- start 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 -- start start 2024-05-17T04:31:48.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113c80 con 0x7f7fa0103f10 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113b00 con 0x7f7fa0108840 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7fa63aa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0113e00 con 0x7f7fa0109190 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7f9ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:48.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7f9ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37132/0 (socket says 172.21.15.50:37132) 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.869+0000 7f7f9ffff700 1 -- 172.21.15.50:0/1567919004 learned_addr learned my addr 172.21.15.50:0/1567919004 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7fa4947700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 -- 172.21.15.50:0/1567919004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0109190 msgr2=0x7f7fa00765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 -- 172.21.15.50:0/1567919004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 msgr2=0x7f7fa0076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f97fff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:48.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 -- 172.21.15.50:0/1567919004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a8950 con 0x7f7fa0103f10 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f97fff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9ffff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000d950 tx=0x7f7f9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90016070 con 0x7f7fa0103f10 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a8be0 con 0x7f7fa0103f10 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f90009dd0 con 0x7f7fa0103f10 2024-05-17T04:31:48.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000f3c0 con 0x7f7fa0103f10 2024-05-17T04:31:48.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.870+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a9170 con 0x7f7fa0103f10 2024-05-17T04:31:48.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.872+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0104320 con 0x7f7fa0103f10 2024-05-17T04:31:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.874+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7f90019070 con 0x7f7fa0103f10 2024-05-17T04:31:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.875+0000 7f7f9dffb700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:48.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.875+0000 7f7f97fff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.875+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f7f900e1d90 con 0x7f7fa0103f10 2024-05-17T04:31:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.876+0000 7f7f97fff700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7fa01073f0 tx=0x7f7f88005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:48.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:48.878+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f900a9b60 con 0x7f7fa0103f10 2024-05-17T04:31:49.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.105+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f7fa0077060 con 0x7f7fa0103f10 2024-05-17T04:31:49.564 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:31:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: 2024-05-17T04:31:49.064+0000 7f1a9f2bb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T04:31:49.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.946+0000 7f7f9dffb700 1 -- 172.21.15.50:0/1567919004 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f7f900a9460 con 0x7f7fa0103f10 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 msgr2=0x7f7f8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7fa01073f0 tx=0x7f7f88005db0 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 msgr2=0x7f7fa0075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000d950 tx=0x7f7f9000dc60 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 shutdown_connections 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fa0109190 0x7f7fa00765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7f8c0b2880 0x7f7f8c0b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7fa0103f10 0x7f7fa0075b30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 --2- 172.21.15.50:0/1567919004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fa0108840 0x7f7fa0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 >> 172.21.15.50:0/1567919004 conn(0x7f7fa00ff9b0 msgr2=0x7f7fa0101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:49.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.949+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 shutdown_connections 2024-05-17T04:31:49.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:49.950+0000 7f7fa63aa700 1 -- 172.21.15.50:0/1567919004 wait complete. 2024-05-17T04:31:50.228 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:49 smithi050 ceph-mon[31739]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T04:31:50.228 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T04:31:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:49 smithi105 ceph-mon[34795]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T04:31:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T04:31:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:49 smithi110 ceph-mon[35497]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-17T04:31:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-17T04:31:50.659 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T04:31:50.920 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:51.203 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:50 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T04:31:51.203 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:50 smithi050 ceph-mon[31739]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T04:31:51.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:50 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T04:31:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:50 smithi105 ceph-mon[34795]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T04:31:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:50 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1567919004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-17T04:31:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:50 smithi110 ceph-mon[35497]: osdmap e79: 12 total, 12 up, 12 in 2024-05-17T04:31:52.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:51 smithi050 ceph-mon[31739]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-17T04:31:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:51 smithi105 ceph-mon[34795]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-17T04:31:52.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:51 smithi110 ceph-mon[35497]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-17T04:31:52.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 -- 172.21.15.50:0/2337507000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 msgr2=0x7f36f4108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:52.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/2337507000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4108c60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36e8009990 tx=0x7f36e8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 -- 172.21.15.50:0/2337507000 shutdown_connections 2024-05-17T04:31:52.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/2337507000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f4110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/2337507000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 0x7f36f4104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/2337507000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4108c60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 -- 172.21.15.50:0/2337507000 >> 172.21.15.50:0/2337507000 conn(0x7f36f40ff9c0 msgr2=0x7f36f4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 -- 172.21.15.50:0/2337507000 shutdown_connections 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.765+0000 7f36fc0a2700 1 -- 172.21.15.50:0/2337507000 wait complete. 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 Processor -- start 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 -- start start 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 0x7f36f4075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f40765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:52.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4113c60 con 0x7f36f4108850 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4113ae0 con 0x7f36f4103f20 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.766+0000 7f36fc0a2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4113de0 con 0x7f36f41091a0 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37152/0 (socket says 172.21.15.50:37152) 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 -- 172.21.15.50:0/1146897369 learned_addr learned my addr 172.21.15.50:0/1146897369 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f9e3e700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 0x7f36f4075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36fa63f700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f40765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 -- 172.21.15.50:0/1146897369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 msgr2=0x7f36f40765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f40765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 -- 172.21.15.50:0/1146897369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 msgr2=0x7f36f4075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:52.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 0x7f36f4075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 -- 172.21.15.50:0/1146897369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f41a8b50 con 0x7f36f4108850 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36fa63f700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f40765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36f963d700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000c00 tx=0x7f36e8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8027070 con 0x7f36f4108850 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36e800fc40 con 0x7f36f4108850 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f41a8de0 con 0x7f36f4108850 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8022bc0 con 0x7f36f4108850 2024-05-17T04:31:52.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.767+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f41a9200 con 0x7f36f4108850 2024-05-17T04:31:52.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.769+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f4104330 con 0x7f36f4108850 2024-05-17T04:31:52.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.770+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f36e802a070 con 0x7f36f4108850 2024-05-17T04:31:52.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.771+0000 7f36e6ffd700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:52.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.771+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f36e80e3470 con 0x7f36f4108850 2024-05-17T04:31:52.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.771+0000 7f36f9e3e700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:52.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.771+0000 7f36f9e3e700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 0x7f36e00b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005950 tx=0x7f36f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.774+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36e805fdf0 con 0x7f36f4108850 2024-05-17T04:31:52.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:52.990+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f36f4077060 con 0x7f36f4108850 2024-05-17T04:31:54.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:53 smithi105 ceph-mon[34795]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:54.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:53 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T04:31:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:53 smithi110 ceph-mon[35497]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:53 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T04:31:54.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:53 smithi050 ceph-mon[31739]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:54.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:53 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-17T04:31:54.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.967+0000 7f36e6ffd700 1 -- 172.21.15.50:0/1146897369 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f36e8013070 con 0x7f36f4108850 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 msgr2=0x7f36e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 0x7f36e00b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005950 tx=0x7f36f000b410 comp rx=0 tx=0).stop 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 msgr2=0x7f36f4076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36e8000c00 tx=0x7f36e8005ef0 comp rx=0 tx=0).stop 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 shutdown_connections 2024-05-17T04:31:54.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36f41091a0 0x7f36f40765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36f4103f20 0x7f36f4075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f36e00b2880 0x7f36e00b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 --2- 172.21.15.50:0/1146897369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36f4108850 0x7f36f4076070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.971+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 >> 172.21.15.50:0/1146897369 conn(0x7f36f40ff9c0 msgr2=0x7f36f4101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.972+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 shutdown_connections 2024-05-17T04:31:54.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:54.972+0000 7f36fc0a2700 1 -- 172.21.15.50:0/1146897369 wait complete. 2024-05-17T04:31:54.975 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' created 2024-05-17T04:31:55.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:54 smithi105 ceph-mon[34795]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T04:31:55.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:54 smithi110 ceph-mon[35497]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T04:31:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:54 smithi050 ceph-mon[31739]: osdmap e80: 12 total, 12 up, 12 in 2024-05-17T04:31:55.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-17T04:31:56.045 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:56.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:55 smithi050 ceph-mon[31739]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:56.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:55 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T04:31:56.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:55 smithi050 ceph-mon[31739]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T04:31:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:55 smithi105 ceph-mon[34795]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:55 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T04:31:56.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:55 smithi105 ceph-mon[34795]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T04:31:56.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:55 smithi110 ceph-mon[35497]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-17T04:31:56.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:55 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1146897369' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-17T04:31:56.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:55 smithi110 ceph-mon[35497]: osdmap e81: 12 total, 12 up, 12 in 2024-05-17T04:31:57.304 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:56 smithi050 ceph-mon[31739]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T04:31:57.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:56 smithi105 ceph-mon[34795]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T04:31:57.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:56 smithi110 ceph-mon[35497]: osdmap e82: 12 total, 12 up, 12 in 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 -- 172.21.15.50:0/1138425487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218104dc0 msgr2=0x7f52181051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 --2- 172.21.15.50:0/1138425487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218104dc0 0x7f52181051d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5208009990 tx=0x7f5208009ca0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 -- 172.21.15.50:0/1138425487 shutdown_connections 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 --2- 172.21.15.50:0/1138425487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521810e220 0x7f5218110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 --2- 172.21.15.50:0/1138425487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218104dc0 0x7f52181051d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 --2- 172.21.15.50:0/1138425487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5218105710 0x7f521810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 -- 172.21.15.50:0/1138425487 >> 172.21.15.50:0/1138425487 conn(0x7f5218079bd0 msgr2=0x7f5218079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 -- 172.21.15.50:0/1138425487 shutdown_connections 2024-05-17T04:31:57.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.756+0000 7f521d177700 1 -- 172.21.15.50:0/1138425487 wait complete. 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.757+0000 7f521d177700 1 Processor -- start 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.757+0000 7f521d177700 1 -- start start 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218104dc0 0x7f521819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f521810e220 0x7f521819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113c70 con 0x7f5218105710 2024-05-17T04:31:57.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113af0 con 0x7f5218104dc0 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521d177700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113df0 con 0x7f521810e220 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37166/0 (socket says 172.21.15.50:37166) 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 -- 172.21.15.50:0/2981699377 learned_addr learned my addr 172.21.15.50:0/2981699377 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f5216d9d700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218104dc0 0x7f521819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521759e700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f521810e220 0x7f521819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 -- 172.21.15.50:0/2981699377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f521810e220 msgr2=0x7f521819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:57.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f521810e220 0x7f521819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 -- 172.21.15.50:0/2981699377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218104dc0 msgr2=0x7f521819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218104dc0 0x7f521819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 -- 172.21.15.50:0/2981699377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521819f5c0 con 0x7f5218105710 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.758+0000 7f521659c700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d980 tx=0x7f520c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.759+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c017070 con 0x7f5218105710 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.759+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f520c005020 con 0x7f5218105710 2024-05-17T04:31:57.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.759+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c00f980 con 0x7f5218105710 2024-05-17T04:31:57.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.759+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521819f8b0 con 0x7f5218105710 2024-05-17T04:31:57.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.759+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181029e0 con 0x7f5218105710 2024-05-17T04:31:57.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.760+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521806c920 con 0x7f5218105710 2024-05-17T04:31:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.761+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f520c004a30 con 0x7f5218105710 2024-05-17T04:31:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.762+0000 7f51fffff700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 0x7f52000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:31:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.762+0000 7f5216d9d700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 0x7f52000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:31:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.762+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f520c0e26c0 con 0x7f5218105710 2024-05-17T04:31:57.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.763+0000 7f5216d9d700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 0x7f52000b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f520800b5c0 tx=0x7f5208005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:31:57.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.766+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f520c020020 con 0x7f5218105710 2024-05-17T04:31:57.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.979+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f52181051d0 con 0x7f5218105710 2024-05-17T04:31:57.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.993+0000 7f51fffff700 1 -- 172.21.15.50:0/2981699377 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f520c0aa160 con 0x7f5218105710 2024-05-17T04:31:57.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.995+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 msgr2=0x7f52000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.995+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 0x7f52000b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f520800b5c0 tx=0x7f5208005960 comp rx=0 tx=0).stop 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.995+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 msgr2=0x7f521819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.995+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f520c00d980 tx=0x7f520c00dc90 comp rx=0 tx=0).stop 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.995+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 shutdown_connections 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f521810e220 0x7f521819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218104dc0 0x7f521819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52000b2880 0x7f52000b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 --2- 172.21.15.50:0/2981699377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5218105710 0x7f521819ac90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:31:57.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 >> 172.21.15.50:0/2981699377 conn(0x7f5218079bd0 msgr2=0x7f52180ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:31:57.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 shutdown_connections 2024-05-17T04:31:57.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:31:57.996+0000 7f521d177700 1 -- 172.21.15.50:0/2981699377 wait complete. 2024-05-17T04:31:57.999 INFO:teuthology.orchestra.run.smithi050.stderr:set pool 4 allow_ec_overwrites to true 2024-05-17T04:31:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:57 smithi105 ceph-mon[34795]: pgmap v301: 129 pgs: 1 creating+activating, 3 creating+peering, 26 unknown, 99 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:57 smithi105 ceph-mon[34795]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:57 smithi105 ceph-mon[34795]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T04:31:58.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:57 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T04:31:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:57 smithi110 ceph-mon[35497]: pgmap v301: 129 pgs: 1 creating+activating, 3 creating+peering, 26 unknown, 99 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:57 smithi110 ceph-mon[35497]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:57 smithi110 ceph-mon[35497]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T04:31:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:57 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T04:31:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:57 smithi050 ceph-mon[31739]: pgmap v301: 129 pgs: 1 creating+activating, 3 creating+peering, 26 unknown, 99 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:31:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:57 smithi050 ceph-mon[31739]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-17T04:31:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:57 smithi050 ceph-mon[31739]: osdmap e83: 12 total, 12 up, 12 in 2024-05-17T04:31:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:57 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-17T04:31:58.865 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-17T04:31:59.129 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:31:59.252 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:58 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T04:31:59.252 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:31:58 smithi050 ceph-mon[31739]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T04:31:59.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:58 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T04:31:59.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:31:58 smithi105 ceph-mon[34795]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T04:31:59.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:58 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2981699377' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-17T04:31:59.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:31:58 smithi110 ceph-mon[35497]: osdmap e84: 12 total, 12 up, 12 in 2024-05-17T04:32:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:00 smithi105 ceph-mon[34795]: pgmap v304: 129 pgs: 1 creating+activating, 3 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:00 smithi110 ceph-mon[35497]: pgmap v304: 129 pgs: 1 creating+activating, 3 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:00 smithi050 ceph-mon[31739]: pgmap v304: 129 pgs: 1 creating+activating, 3 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:00.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 -- 172.21.15.50:0/2801697148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c107980 msgr2=0x7f969c107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:00.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 --2- 172.21.15.50:0/2801697148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c107980 0x7f969c107d90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f968c009990 tx=0x7f968c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 -- 172.21.15.50:0/2801697148 shutdown_connections 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 --2- 172.21.15.50:0/2801697148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c1082d0 0x7f969c10fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 --2- 172.21.15.50:0/2801697148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c103570 0x7f969c103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 --2- 172.21.15.50:0/2801697148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c107980 0x7f969c107d90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 -- 172.21.15.50:0/2801697148 >> 172.21.15.50:0/2801697148 conn(0x7f969c0fe1a0 msgr2=0x7f969c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 -- 172.21.15.50:0/2801697148 shutdown_connections 2024-05-17T04:32:00.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.946+0000 7f96a45d2700 1 -- 172.21.15.50:0/2801697148 wait complete. 2024-05-17T04:32:00.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 Processor -- start 2024-05-17T04:32:00.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 -- start start 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c107980 0x7f969c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c1082d0 0x7f969c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112da0 con 0x7f969c103570 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112c20 con 0x7f969c1082d0 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a45d2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f969c112f20 con 0x7f969c107980 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a236e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a236e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49878/0 (socket says 172.21.15.50:49878) 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a236e700 1 -- 172.21.15.50:0/301038181 learned_addr learned my addr 172.21.15.50:0/301038181 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:00.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a2b6f700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c1082d0 0x7f969c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.947+0000 7f96a1b6d700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c107980 0x7f969c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 -- 172.21.15.50:0/301038181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c107980 msgr2=0x7f969c19bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c107980 0x7f969c19bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 -- 172.21.15.50:0/301038181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c1082d0 msgr2=0x7f969c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c1082d0 0x7f969c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 -- 172.21.15.50:0/301038181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c1a6a50 con 0x7f969c103570 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a236e700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f969800d980 tx=0x7f969800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698016070 con 0x7f969c103570 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9698004830 con 0x7f969c103570 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c1a6d40 con 0x7f969c103570 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698005440 con 0x7f969c103570 2024-05-17T04:32:00.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.948+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c1a7220 con 0x7f969c103570 2024-05-17T04:32:00.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.949+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969c103980 con 0x7f969c103570 2024-05-17T04:32:00.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.951+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f96980055e0 con 0x7f969c103570 2024-05-17T04:32:00.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.952+0000 7f96937fe700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 0x7f96880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:00.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.952+0000 7f96a1b6d700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 0x7f96880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:00.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.952+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f96980e2740 con 0x7f969c103570 2024-05-17T04:32:00.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.952+0000 7f96a1b6d700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 0x7f96880b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f968c006010 tx=0x7f968c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:00.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:00.957+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f969801f020 con 0x7f969c103570 2024-05-17T04:32:01.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:01.243+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f969c06be20 con 0x7f969c103570 2024-05-17T04:32:02.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.015+0000 7f96937fe700 1 -- 172.21.15.50:0/301038181 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f96980aa1e0 con 0x7f969c103570 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 msgr2=0x7f96880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 0x7f96880b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f968c006010 tx=0x7f968c00b540 comp rx=0 tx=0).stop 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 msgr2=0x7f969c19b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f969800d980 tx=0x7f969800dc90 comp rx=0 tx=0).stop 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 shutdown_connections 2024-05-17T04:32:02.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f969c1082d0 0x7f969c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f96880b2880 0x7f96880b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f969c103570 0x7f969c19b790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 --2- 172.21.15.50:0/301038181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f969c107980 0x7f969c19bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 >> 172.21.15.50:0/301038181 conn(0x7f969c0fe1a0 msgr2=0x7f969c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 shutdown_connections 2024-05-17T04:32:02.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:02.019+0000 7f96a45d2700 1 -- 172.21.15.50:0/301038181 wait complete. 2024-05-17T04:32:02.021 INFO:teuthology.orchestra.run.smithi050.stderr:added data pool 4 to fsmap 2024-05-17T04:32:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:02 smithi105 ceph-mon[34795]: pgmap v305: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:02.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:02 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T04:32:02.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:02 smithi110 ceph-mon[35497]: pgmap v305: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:02 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T04:32:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:02 smithi050 ceph-mon[31739]: pgmap v305: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:02 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-17T04:32:02.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:32:03.231 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:03.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:03 smithi105 ceph-mon[34795]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T04:32:03.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:03 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T04:32:03.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:03 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:03 smithi110 ceph-mon[35497]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T04:32:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:03 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T04:32:03.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:03 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:03.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:03 smithi050 ceph-mon[31739]: osdmap e85: 12 total, 12 up, 12 in 2024-05-17T04:32:03.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:03 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/301038181' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-17T04:32:03.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:03 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:04 smithi105 ceph-mon[34795]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:04 smithi105 ceph-mon[34795]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T04:32:04.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:04 smithi105 ceph-mon[34795]: Cluster is now healthy 2024-05-17T04:32:04.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:04 smithi110 ceph-mon[35497]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:04.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:04 smithi110 ceph-mon[35497]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T04:32:04.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:04 smithi110 ceph-mon[35497]: Cluster is now healthy 2024-05-17T04:32:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:04 smithi050 ceph-mon[31739]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:04 smithi050 ceph-mon[31739]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-17T04:32:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:04 smithi050 ceph-mon[31739]: Cluster is now healthy 2024-05-17T04:32:05.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 -- 172.21.15.50:0/1491996100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d4102e70 msgr2=0x7f52d4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:05.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 --2- 172.21.15.50:0/1491996100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d4102e70 0x7f52d4103280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009990 tx=0x7f52c4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 -- 172.21.15.50:0/1491996100 shutdown_connections 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 --2- 172.21.15.50:0/1491996100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d41080a0 0x7f52d410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 --2- 172.21.15.50:0/1491996100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d4102e70 0x7f52d4103280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 --2- 172.21.15.50:0/1491996100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d4107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 -- 172.21.15.50:0/1491996100 >> 172.21.15.50:0/1491996100 conn(0x7f52d4079ae0 msgr2=0x7f52d4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 -- 172.21.15.50:0/1491996100 shutdown_connections 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.140+0000 7f52d91e1700 1 -- 172.21.15.50:0/1491996100 wait complete. 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 Processor -- start 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 -- start start 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d4102e70 0x7f52d419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:05.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4112b80 con 0x7f52d41080a0 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4112a00 con 0x7f52d4102e70 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d91e1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4112d00 con 0x7f52d4107750 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49882/0 (socket says 172.21.15.50:49882) 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d359e700 1 -- 172.21.15.50:0/592186891 learned_addr learned my addr 172.21.15.50:0/592186891 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d2d9d700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d4102e70 0x7f52d419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.141+0000 7f52d259c700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 -- 172.21.15.50:0/592186891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 msgr2=0x7f52d419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 -- 172.21.15.50:0/592186891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d4102e70 msgr2=0x7f52d419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d4102e70 0x7f52d419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 -- 172.21.15.50:0/592186891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d419f570 con 0x7f52d41080a0 2024-05-17T04:32:05.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d259c700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d359e700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52cc007ee0 tx=0x7f52cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00dbd0 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d419f860 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52cc00dd70 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0058d0 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.142+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d419fda0 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.143+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4103280 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.145+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f52cc01a020 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.146+0000 7f52bbfff700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 0x7f52bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.146+0000 7f52d2d9d700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 0x7f52bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.146+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f52cc0e2250 con 0x7f52d41080a0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.146+0000 7f52d2d9d700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 0x7f52bc0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098e0 tx=0x7f52c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:05.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.149+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52cc010070 con 0x7f52d41080a0 2024-05-17T04:32:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.411+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d402ea80 con 0x7f52d41080a0 2024-05-17T04:32:05.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.411+0000 7f52bbfff700 1 -- 172.21.15.50:0/592186891 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f52cc0a9cd0 con 0x7f52d41080a0 2024-05-17T04:32:05.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:05.412 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:01.245437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 msgr2=0x7f52bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 0x7f52bc0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098e0 tx=0x7f52c4005960 comp rx=0 tx=0).stop 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 msgr2=0x7f52d419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52cc007ee0 tx=0x7f52cc00d3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 shutdown_connections 2024-05-17T04:32:05.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f52bc0b2880 0x7f52bc0b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52d41080a0 0x7f52d419ef80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52d4102e70 0x7f52d419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 --2- 172.21.15.50:0/592186891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d4107750 0x7f52d419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 >> 172.21.15.50:0/592186891 conn(0x7f52d4079ae0 msgr2=0x7f52d406c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 shutdown_connections 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:05.414+0000 7f52d91e1700 1 -- 172.21.15.50:0/592186891 wait complete. 2024-05-17T04:32:05.415 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-05-17T04:32:06.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:06 smithi050 ceph-mon[31739]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:06.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:06 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/592186891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:06.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:32:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:06 smithi105 ceph-mon[34795]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:06.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:06 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/592186891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:06.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:06 smithi110 ceph-mon[35497]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:06.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:06 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/592186891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:08 smithi050 ceph-mon[31739]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:08 smithi105 ceph-mon[34795]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:08 smithi110 ceph-mon[35497]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.368+0000 7f5b3d937700 1 -- 172.21.15.50:0/3949172624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 msgr2=0x7f5b3810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.368+0000 7f5b3d937700 1 --2- 172.21.15.50:0/3949172624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3810fa00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009990 tx=0x7f5b30009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 -- 172.21.15.50:0/3949172624 shutdown_connections 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 --2- 172.21.15.50:0/3949172624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3810fa00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 --2- 172.21.15.50:0/3949172624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b38103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 --2- 172.21.15.50:0/3949172624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 0x7f5b38107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 -- 172.21.15.50:0/3949172624 >> 172.21.15.50:0/3949172624 conn(0x7f5b38079ae0 msgr2=0x7f5b38079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 -- 172.21.15.50:0/3949172624 shutdown_connections 2024-05-17T04:32:08.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 -- 172.21.15.50:0/3949172624 wait complete. 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.369+0000 7f5b3d937700 1 Processor -- start 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 -- start start 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b3819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 0x7f5b3819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38113d00 con 0x7f5b381080a0 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38113b80 con 0x7f5b38102e70 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b3d937700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38113e80 con 0x7f5b38107750 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49902/0 (socket says 172.21.15.50:49902) 2024-05-17T04:32:08.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 -- 172.21.15.50:0/966612819 learned_addr learned my addr 172.21.15.50:0/966612819 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b367fc700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 0x7f5b3819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b36ffd700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b3819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 -- 172.21.15.50:0/966612819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 msgr2=0x7f5b3819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 0x7f5b3819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 -- 172.21.15.50:0/966612819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 msgr2=0x7f5b3819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b3819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b377fe700 1 -- 172.21.15.50:0/966612819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3819f5f0 con 0x7f5b381080a0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.370+0000 7f5b36ffd700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b3819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b377fe700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b300057c0 tx=0x7f5b300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30027070 con 0x7f5b381080a0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3819f880 con 0x7f5b381080a0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b3000fb00 con 0x7f5b381080a0 2024-05-17T04:32:08.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30022c60 con 0x7f5b381080a0 2024-05-17T04:32:08.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.371+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3819fd60 con 0x7f5b381080a0 2024-05-17T04:32:08.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.372+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b38103280 con 0x7f5b381080a0 2024-05-17T04:32:08.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.373+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5b3002a070 con 0x7f5b381080a0 2024-05-17T04:32:08.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.374+0000 7f5b3c935700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:08.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.374+0000 7f5b36ffd700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:08.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.374+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f5b30022e00 con 0x7f5b381080a0 2024-05-17T04:32:08.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.375+0000 7f5b36ffd700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 0x7f5b200b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009dd0 tx=0x7f5b28009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:08.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.380+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b30013070 con 0x7f5b381080a0 2024-05-17T04:32:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.595+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b3819ba80 con 0x7f5b381080a0 2024-05-17T04:32:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.597+0000 7f5b3c935700 1 -- 172.21.15.50:0/966612819 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24103 (secure 0 0 0) 0x7f5b300ab240 con 0x7f5b381080a0 2024-05-17T04:32:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:08.597 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:01.245212+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 msgr2=0x7f5b200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 0x7f5b200b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009dd0 tx=0x7f5b28009040 comp rx=0 tx=0).stop 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 msgr2=0x7f5b3819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5b300057c0 tx=0x7f5b300057f0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 shutdown_connections 2024-05-17T04:32:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b200b2880 0x7f5b200b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b381080a0 0x7f5b3819ef50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b38102e70 0x7f5b3819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 --2- 172.21.15.50:0/966612819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b38107750 0x7f5b3819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 >> 172.21.15.50:0/966612819 conn(0x7f5b38079ae0 msgr2=0x7f5b381053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 shutdown_connections 2024-05-17T04:32:08.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:08.599+0000 7f5b3d937700 1 -- 172.21.15.50:0/966612819 wait complete. 2024-05-17T04:32:09.291 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:09 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/966612819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:09.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:09 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/966612819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:09.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:09 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/966612819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:09.321 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:32:09.574 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:10 smithi105 ceph-mon[34795]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:10 smithi110 ceph-mon[35497]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:10 smithi050 ceph-mon[31739]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.279+0000 7fb493bc4700 1 -- 172.21.15.50:0/2579318421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c107730 msgr2=0x7fb48c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.279+0000 7fb493bc4700 1 --2- 172.21.15.50:0/2579318421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c107730 0x7fb48c107b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb480009990 tx=0x7fb480009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.279+0000 7fb493bc4700 1 -- 172.21.15.50:0/2579318421 shutdown_connections 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.279+0000 7fb493bc4700 1 --2- 172.21.15.50:0/2579318421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c108080 0x7fb48c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 --2- 172.21.15.50:0/2579318421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 0x7fb48c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 --2- 172.21.15.50:0/2579318421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c107730 0x7fb48c107b40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 -- 172.21.15.50:0/2579318421 >> 172.21.15.50:0/2579318421 conn(0x7fb48c079bd0 msgr2=0x7fb48c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 -- 172.21.15.50:0/2579318421 shutdown_connections 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 -- 172.21.15.50:0/2579318421 wait complete. 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 Processor -- start 2024-05-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.280+0000 7fb493bc4700 1 -- start start 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 0x7fb48c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c107730 0x7fb48c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c112b40 con 0x7fb48c108080 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1129c0 con 0x7fb48c102e00 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb493bc4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c112cc0 con 0x7fb48c107730 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52388/0 (socket says 172.21.15.50:52388) 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb491960700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 0x7fb48c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 -- 172.21.15.50:0/497423386 learned_addr learned my addr 172.21.15.50:0/497423386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb49115f700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c107730 0x7fb48c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 -- 172.21.15.50:0/497423386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c107730 msgr2=0x7fb48c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c107730 0x7fb48c199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 -- 172.21.15.50:0/497423386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 msgr2=0x7fb48c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 0x7fb48c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 -- 172.21.15.50:0/497423386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c19e460 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.281+0000 7fb492161700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb484007d70 tx=0x7fb48400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.282+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484016020 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.282+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb48400ac90 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.282+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c19e750 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.282+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4840115e0 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.282+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c19ec60 con 0x7fb48c108080 2024-05-17T04:32:11.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.283+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c103210 con 0x7fb48c108080 2024-05-17T04:32:11.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.284+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb484019020 con 0x7fb48c108080 2024-05-17T04:32:11.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.285+0000 7fb47effd700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 0x7fb4780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:11.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.285+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fb4840e2840 con 0x7fb48c108080 2024-05-17T04:32:11.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.285+0000 7fb491960700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 0x7fb4780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:11.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.286+0000 7fb491960700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 0x7fb4780b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb488005950 tx=0x7fb48800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:11.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.288+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb484020020 con 0x7fb48c108080 2024-05-17T04:32:11.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.504+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb48c02ea80 con 0x7fb48c108080 2024-05-17T04:32:11.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.506+0000 7fb47effd700 1 -- 172.21.15.50:0/497423386 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24103 (secure 0 0 0) 0x7fb4840aa320 con 0x7fb48c108080 2024-05-17T04:32:11.507 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:01.245212+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 msgr2=0x7fb4780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 0x7fb4780b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb488005950 tx=0x7fb48800a300 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 msgr2=0x7fb48c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb484007d70 tx=0x7fb48400a340 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 shutdown_connections 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4780b2900 0x7fb4780b4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb48c108080 0x7fb48c19de70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb48c102e00 0x7fb48c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 --2- 172.21.15.50:0/497423386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c107730 0x7fb48c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:11.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.508+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 >> 172.21.15.50:0/497423386 conn(0x7fb48c079bd0 msgr2=0x7fb48c105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:11.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.509+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 shutdown_connections 2024-05-17T04:32:11.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:11.509+0000 7fb493bc4700 1 -- 172.21.15.50:0/497423386 wait complete. 2024-05-17T04:32:12.203 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:12 smithi050 ceph-mon[31739]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:12.204 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:12 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/497423386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:12.250 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:32:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:12 smithi105 ceph-mon[34795]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:12.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:12 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/497423386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:12 smithi110 ceph-mon[35497]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:12 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/497423386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:12.510 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:14.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 -- 172.21.15.50:0/2270729815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 msgr2=0x7f88e4107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:14.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 --2- 172.21.15.50:0/2270729815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e4107b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009990 tx=0x7f88d8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.265 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:14 smithi050 ceph-mon[31739]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 -- 172.21.15.50:0/2270729815 shutdown_connections 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 --2- 172.21.15.50:0/2270729815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e41080a0 0x7f88e410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 --2- 172.21.15.50:0/2270729815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e4102e70 0x7f88e4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 --2- 172.21.15.50:0/2270729815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e4107b60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 -- 172.21.15.50:0/2270729815 >> 172.21.15.50:0/2270729815 conn(0x7f88e4079ae0 msgr2=0x7f88e4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 -- 172.21.15.50:0/2270729815 shutdown_connections 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.262+0000 7f88e8c57700 1 -- 172.21.15.50:0/2270729815 wait complete. 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.263+0000 7f88e8c57700 1 Processor -- start 2024-05-17T04:32:14.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.263+0000 7f88e8c57700 1 -- start start 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 0x7f88e419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e41080a0 0x7f88e419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4112b60 con 0x7f88e4107750 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41129e0 con 0x7f88e41080a0 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e8c57700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4112ce0 con 0x7f88e4102e70 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e1d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e1d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52414/0 (socket says 172.21.15.50:52414) 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e1d9b700 1 -- 172.21.15.50:0/984668306 learned_addr learned my addr 172.21.15.50:0/984668306 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e259c700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 0x7f88e419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.264+0000 7f88e2d9d700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e41080a0 0x7f88e419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:14.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 -- 172.21.15.50:0/984668306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 msgr2=0x7f88e419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 0x7f88e419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 -- 172.21.15.50:0/984668306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e41080a0 msgr2=0x7f88e419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e41080a0 0x7f88e419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 -- 172.21.15.50:0/984668306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e419f5a0 con 0x7f88e4107750 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e259c700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 0x7f88e419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e1d9b700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8027070 con 0x7f88e4107750 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e419f830 con 0x7f88e4107750 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a71b0 con 0x7f88e4107750 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88d800fbc0 con 0x7f88e4107750 2024-05-17T04:32:14.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.265+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8022ac0 con 0x7f88e4107750 2024-05-17T04:32:14.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.266+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4005330 con 0x7f88e4107750 2024-05-17T04:32:14.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.268+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f88d802a070 con 0x7f88e4107750 2024-05-17T04:32:14.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.269+0000 7f88d37fe700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:14.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.269+0000 7f88e259c700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:14.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.269+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f88d80e3530 con 0x7f88e4107750 2024-05-17T04:32:14.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.269+0000 7f88e259c700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 0x7f88cc0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88d4005950 tx=0x7f88d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:14.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.272+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88d805fdf0 con 0x7f88e4107750 2024-05-17T04:32:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:14 smithi105 ceph-mon[34795]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:14.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:14 smithi110 ceph-mon[35497]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:14.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.534+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88c4005a40 con 0x7f88e4107750 2024-05-17T04:32:14.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.535+0000 7f88d37fe700 1 -- 172.21.15.50:0/984668306 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f88d8013070 con 0x7f88e4107750 2024-05-17T04:32:14.536 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:14.536 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:01.245437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 msgr2=0x7f88cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 0x7f88cc0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88d4005950 tx=0x7f88d400b410 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 msgr2=0x7f88e419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8005ef0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 shutdown_connections 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88e41080a0 0x7f88e419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88e4102e70 0x7f88e419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88cc0b2880 0x7f88cc0b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.537+0000 7f88e8c57700 1 --2- 172.21.15.50:0/984668306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88e4107750 0x7f88e419ac70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.538+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 >> 172.21.15.50:0/984668306 conn(0x7f88e4079ae0 msgr2=0x7f88e406c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.538+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 shutdown_connections 2024-05-17T04:32:14.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:14.538+0000 7f88e8c57700 1 -- 172.21.15.50:0/984668306 wait complete. 2024-05-17T04:32:14.540 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 17 2024-05-17T04:32:15.139 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:15 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/984668306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:15.168 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:32:15.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:15 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/984668306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:15.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:15 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/984668306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:32:15.421 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:16 smithi050 ceph-mon[31739]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:16 smithi105 ceph-mon[34795]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:16 smithi110 ceph-mon[35497]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 -- 172.21.15.50:0/968797472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444104e10 msgr2=0x7fb444105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 --2- 172.21.15.50:0/968797472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444104e10 0x7fb444105220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb434009990 tx=0x7fb434009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 -- 172.21.15.50:0/968797472 shutdown_connections 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 --2- 172.21.15.50:0/968797472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 0x7fb444110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 --2- 172.21.15.50:0/968797472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444104e10 0x7fb444105220 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 --2- 172.21.15.50:0/968797472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444105760 0x7fb44410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 -- 172.21.15.50:0/968797472 >> 172.21.15.50:0/968797472 conn(0x7fb444079ae0 msgr2=0x7fb444079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 -- 172.21.15.50:0/968797472 shutdown_connections 2024-05-17T04:32:17.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.153+0000 7fb4495f5700 1 -- 172.21.15.50:0/968797472 wait complete. 2024-05-17T04:32:17.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.154+0000 7fb4495f5700 1 Processor -- start 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.154+0000 7fb4495f5700 1 -- start start 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 0x7fb44419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 0x7fb44419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb444113c70 con 0x7fb444105760 2024-05-17T04:32:17.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb444113af0 con 0x7fb444104e10 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4495f5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb444113df0 con 0x7fb44410e240 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52432/0 (socket says 172.21.15.50:52432) 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 -- 172.21.15.50:0/2564669014 learned_addr learned my addr 172.21.15.50:0/2564669014 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4437fe700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 0x7fb44419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb442ffd700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 0x7fb44419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 -- 172.21.15.50:0/2564669014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 msgr2=0x7fb44419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 0x7fb44419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 -- 172.21.15.50:0/2564669014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 msgr2=0x7fb44419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 0x7fb44419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 -- 172.21.15.50:0/2564669014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44419f510 con 0x7fb444105760 2024-05-17T04:32:17.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb442ffd700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 0x7fb44419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.155+0000 7fb4427fc700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c9b0 tx=0x7fb43800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.156+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438007b00 con 0x7fb444105760 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.156+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb438007ca0 con 0x7fb444105760 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.156+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438018660 con 0x7fb444105760 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.156+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44419f800 con 0x7fb444105760 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.156+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44419fcf0 con 0x7fb444105760 2024-05-17T04:32:17.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.157+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444105220 con 0x7fb444105760 2024-05-17T04:32:17.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.159+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb43800f450 con 0x7fb444105760 2024-05-17T04:32:17.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.159+0000 7fb42bfff700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 0x7fb42c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:17.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.160+0000 7fb442ffd700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 0x7fb42c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:17.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.160+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fb4380e2720 con 0x7fb444105760 2024-05-17T04:32:17.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.160+0000 7fb442ffd700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 0x7fb42c0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434009750 tx=0x7fb43400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:17.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.163+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb438014030 con 0x7fb444105760 2024-05-17T04:32:17.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.377+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb44402ea80 con 0x7fb444105760 2024-05-17T04:32:17.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.379+0000 7fb42bfff700 1 -- 172.21.15.50:0/2564669014 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24103 (secure 0 0 0) 0x7fb4380aa230 con 0x7fb444105760 2024-05-17T04:32:17.380 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:17.380 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":85,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:01.245212+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 msgr2=0x7fb42c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 0x7fb42c0b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb434009750 tx=0x7fb43400b580 comp rx=0 tx=0).stop 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 msgr2=0x7fb44419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c9b0 tx=0x7fb43800ccc0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 shutdown_connections 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb44410e240 0x7fb44419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb444104e10 0x7fb44419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb42c0b2880 0x7fb42c0b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 --2- 172.21.15.50:0/2564669014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb444105760 0x7fb44419ac90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:17.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 >> 172.21.15.50:0/2564669014 conn(0x7fb444079ae0 msgr2=0x7fb4440ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:17.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 shutdown_connections 2024-05-17T04:32:17.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:17.381+0000 7fb4495f5700 1 -- 172.21.15.50:0/2564669014 wait complete. 2024-05-17T04:32:18.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T04:31:45.150094+0000', 'modified': '2024-05-17T04:32:01.245437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/3722840784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 3722840784}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 3722840784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24506, 'qdb_cluster': [24506]} 2024-05-17T04:32:18.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T04:32:18.133 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T04:32:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:18 smithi105 ceph-mon[34795]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:18.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:18 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2564669014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:18 smithi110 ceph-mon[35497]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:18 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2564669014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:18 smithi050 ceph-mon[31739]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:18 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2564669014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:18.391 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-05-17T04:32:19.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-17T04:32:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-17T04:32:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-05-17T04:32:19.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-17T04:32:19.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-05-17T04:32:19.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-17T04:32:20.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 -- 172.21.15.50:0/3245746057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 msgr2=0x7fae7810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 --2- 172.21.15.50:0/3245746057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7810fa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b0a0 tx=0x7fae7000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 -- 172.21.15.50:0/3245746057 shutdown_connections 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 --2- 172.21.15.50:0/3245746057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7810fa00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 --2- 172.21.15.50:0/3245746057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 0x7fae78103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 --2- 172.21.15.50:0/3245746057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 0x7fae78107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 -- 172.21.15.50:0/3245746057 >> 172.21.15.50:0/3245746057 conn(0x7fae78079ae0 msgr2=0x7fae78079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 -- 172.21.15.50:0/3245746057 shutdown_connections 2024-05-17T04:32:20.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.161+0000 7fae7ec99700 1 -- 172.21.15.50:0/3245746057 wait complete. 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T04:32:20.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:20 smithi050 ceph-mon[31739]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.162+0000 7fae7ec99700 1 Processor -- start 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.162+0000 7fae7ec99700 1 -- start start 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 0x7fae7819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 0x7fae7819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae78112bc0 con 0x7fae781080a0 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae78112a40 con 0x7fae78102e70 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ec99700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae78112d40 con 0x7fae78107750 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39568/0 (socket says 172.21.15.50:39568) 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 -- 172.21.15.50:0/1923990132 learned_addr learned my addr 172.21.15.50:0/1923990132 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae77fff700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 0x7fae7819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7ca35700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 0x7fae7819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 -- 172.21.15.50:0/1923990132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 msgr2=0x7fae7819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 0x7fae7819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 -- 172.21.15.50:0/1923990132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 msgr2=0x7fae7819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 0x7fae7819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 -- 172.21.15.50:0/1923990132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae781a6840 con 0x7fae781080a0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.163+0000 7fae7d236700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b830 tx=0x7fae70004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.164+0000 7fae75ffb700 1 -- 172.21.15.50:0/1923990132 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70021070 con 0x7fae781080a0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.164+0000 7fae75ffb700 1 -- 172.21.15.50:0/1923990132 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae70007b80 con 0x7fae781080a0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.164+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae781a6ad0 con 0x7fae781080a0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.164+0000 7fae75ffb700 1 -- 172.21.15.50:0/1923990132 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700052a0 con 0x7fae781080a0 2024-05-17T04:32:20.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.164+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae781a6f80 con 0x7fae781080a0 2024-05-17T04:32:20.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.167+0000 7fae75ffb700 1 -- 172.21.15.50:0/1923990132 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fae70024070 con 0x7fae781080a0 2024-05-17T04:32:20.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.168+0000 7fae75ffb700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 0x7fae680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.168+0000 7fae7ca35700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 0x7fae680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.168+0000 7fae75ffb700 1 -- 172.21.15.50:0/1923990132 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fae7000f020 con 0x7fae781080a0 2024-05-17T04:32:20.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.168+0000 7fae7ca35700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 0x7fae680b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004630 tx=0x7fae6c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 Processor -- start 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 -- start start 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e0ab0 0x5654448e0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56544480f080 con 0x5654448e63b0 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5654447fec20 con 0x5654447629a0 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae8d674b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x565444802c60 con 0x5654448e0ab0 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39572/0 (socket says 172.21.15.50:39572) 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.169+0000 7fae77fff700 1 -- 172.21.15.50:0/1602494284 learned_addr learned my addr 172.21.15.50:0/1602494284 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:20.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae7d236700 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e0ab0 0x5654448e0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae7ca35700 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 -- 172.21.15.50:0/1602494284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e0ab0 msgr2=0x5654448e0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e0ab0 0x5654448e0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 -- 172.21.15.50:0/1602494284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 msgr2=0x5654448e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 -- 172.21.15.50:0/1602494284 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565444798570 con 0x5654448e63b0 2024-05-17T04:32:20.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae77fff700 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae6400abf0 tx=0x7fae6400af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f70fdb9e9b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae62ffd700 1 -- 172.21.15.50:0/1602494284 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6401d070 con 0x5654448e63b0 2024-05-17T04:32:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae62ffd700 1 -- 172.21.15.50:0/1602494284 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae6400fd80 con 0x5654448e63b0 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.170+0000 7fae62ffd700 1 -- 172.21.15.50:0/1602494284 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64010650 con 0x5654448e63b0 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- 172.21.15.50:0/1602494284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 msgr2=0x5654448e0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae6400abf0 tx=0x7fae6400af00 comp rx=0 tx=0).stop 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- 172.21.15.50:0/1602494284 shutdown_connections 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e0ab0 0x5654448e0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 --2- 172.21.15.50:0/1602494284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e0570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- 172.21.15.50:0/1602494284 >> 172.21.15.50:0/1602494284 conn(0x5654447c2780 msgr2=0x5654448db170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- 172.21.15.50:0/1602494284 shutdown_connections 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- 172.21.15.50:0/1602494284 wait complete. 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.171+0000 7fae8d674b80 1 -- start start 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 Processor -- start 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x565444706100 con 0x5654448e0ab0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5654447a08f0 con 0x5654447629a0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654448ecd80 con 0x5654448e63b0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae8d674b80 10 client.0 _ll_register_callbacks cb 0x565444795b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39586/0 (socket says 172.21.15.50:39586) 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae77fff700 1 -- 172.21.15.50:0/2719376664 learned_addr learned my addr 172.21.15.50:0/2719376664 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae7ca35700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.172+0000 7fae7d236700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 -- 172.21.15.50:0/2719376664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 msgr2=0x5654448ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448ea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 -- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 msgr2=0x5654448e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 -- 172.21.15.50:0/2719376664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544474f430 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae77fff700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae64005750 tx=0x7fae64005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ec9a562761fd72 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae57fff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64025020 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fae64004950 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae6400fd80 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae64030050 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae8d674b80 10 client.14727 fetch_fsmap learned FSMap version 17 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5654448f7850 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.173+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae6400fd80 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fae64020030 con 0x5654448e0ab0 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 10 client.14727 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 20 client.14727 populate_metadata read hostname 'smithi050' 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654448f9ab0 0x5654448faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 4 client.14727 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:32:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565444810fd0 con 0x5654448f9ab0 2024-05-17T04:32:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.174+0000 7fae7ca35700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654448f9ab0 0x5654448faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.175+0000 7fae7ca35700 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654448f9ab0 0x5654448faf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e6236a4ca93aa5 server_cookie=a56f4f9147e6e9cc in_seq=0 out_seq=0 2024-05-17T04:32:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.175+0000 7fae57fff700 10 client.14727 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.175+0000 7fae57fff700 1 -- 172.21.15.50:0/2719376664 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565444810fd0 con 0x5654448f9ab0 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.175+0000 7fae57fff700 10 client.14727 handle_command_reply: tid=0 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.175+0000 7fae8d674b80 1 client.14727 shutdown 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 msgr2=0x5654448e9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae64005750 tx=0x7fae64005780 comp rx=0 tx=0).stop 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 shutdown_connections 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654448f9ab0 0x5654448faf20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x5654448e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 --2- 172.21.15.50:0/2719376664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448e9f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 >> 172.21.15.50:0/2719376664 conn(0x5654447c2780 msgr2=0x5654448db110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae57fff700 0 client.14727 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:32:20.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 shutdown_connections 2024-05-17T04:32:20.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 1 -- 172.21.15.50:0/2719376664 wait complete. 2024-05-17T04:32:20.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.176+0000 7fae8d674b80 20 client.14727 trim_cache size 0 max 16384 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 Processor -- start 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 -- start start 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654447629a0 0x5654448e0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654448e0ab0 0x5654448f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5654448f9260 con 0x5654448e63b0 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5654448f9930 con 0x5654448e0ab0 2024-05-17T04:32:20.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae8d674b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654448fa000 con 0x5654447629a0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39594/0 (socket says 172.21.15.50:39594) 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 -- 172.21.15.50:0/176388646 learned_addr learned my addr 172.21.15.50:0/176388646 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7ca35700 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654447629a0 0x5654448e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae77fff700 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654448e0ab0 0x5654448f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 -- 172.21.15.50:0/176388646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654447629a0 msgr2=0x5654448e0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654447629a0 0x5654448e0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 -- 172.21.15.50:0/176388646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654448e0ab0 msgr2=0x5654448f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654448e0ab0 0x5654448f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.200+0000 7fae7d236700 1 -- 172.21.15.50:0/176388646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654448ec810 con 0x5654448e63b0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.201+0000 7fae7d236700 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae700e11b0 tx=0x7fae70009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e8ecd59c8910af server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.201+0000 7fae627fc700 1 -- 172.21.15.50:0/176388646 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700aadc0 con 0x5654448e63b0 2024-05-17T04:32:20.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.201+0000 7fae627fc700 1 -- 172.21.15.50:0/176388646 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae700c1030 con 0x5654448e63b0 2024-05-17T04:32:20.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.201+0000 7fae627fc700 1 -- 172.21.15.50:0/176388646 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700244e0 con 0x5654448e63b0 2024-05-17T04:32:20.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- 172.21.15.50:0/176388646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 msgr2=0x5654448e9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae700e11b0 tx=0x7fae70009040 comp rx=0 tx=0).stop 2024-05-17T04:32:20.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- 172.21.15.50:0/176388646 shutdown_connections 2024-05-17T04:32:20.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e63b0 0x5654448e9f10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654447629a0 0x5654448e0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 --2- 172.21.15.50:0/176388646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654448e0ab0 0x5654448f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- 172.21.15.50:0/176388646 >> 172.21.15.50:0/176388646 conn(0x5654447c2780 msgr2=0x5654448db110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- 172.21.15.50:0/176388646 shutdown_connections 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- 172.21.15.50:0/176388646 wait complete. 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 -- start start 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.202+0000 7fae8d674b80 1 Processor -- start 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x565444904950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5654448fada0 con 0x5654448e0ab0 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5654448fa6d0 con 0x5654447629a0 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654448fb470 con 0x5654448e63b0 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae8d674b80 10 client.0 _ll_register_callbacks cb 0x565444795b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39602/0 (socket says 172.21.15.50:39602) 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 -- 172.21.15.50:0/292750205 learned_addr learned my addr 172.21.15.50:0/292750205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae7d236700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae7ca35700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x565444904950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 -- 172.21.15.50:0/292750205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 msgr2=0x5654448dfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448dfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 -- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 msgr2=0x565444904950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x565444904950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 -- 172.21.15.50:0/292750205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654448e0330 con 0x5654448e0ab0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae77fff700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5654448eb180 tx=0x7fae64010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a995d1f4ca97d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.203+0000 7fae57fff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64025070 con 0x5654448e0ab0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fae64018640 con 0x5654448e0ab0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae700c1030 con 0x5654448e0ab0 2024-05-17T04:32:20.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae6401d040 con 0x5654448e0ab0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 10 client.14733 fetch_fsmap learned FSMap version 17 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x565444964f80 con 0x5654448e0ab0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae700c1030 con 0x5654448e0ab0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fae64020040 con 0x5654448e0ab0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 10 client.14733 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 20 client.14733 populate_metadata read hostname 'smithi050' 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654449673c0 0x565444968830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 4 client.14733 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.204+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "status"}) -- 0x56544480a7b0 con 0x5654449673c0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.205+0000 7fae7ca35700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654449673c0 0x565444968830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.205+0000 7fae57fff700 10 client.14733 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.205+0000 7fae7ca35700 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654449673c0 0x565444968830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5c64fdd866a106 server_cookie=8556718347152d2e in_seq=0 out_seq=0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.205+0000 7fae57fff700 1 -- 172.21.15.50:0/292750205 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56544480a7b0 con 0x5654449673c0 2024-05-17T04:32:20.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.205+0000 7fae57fff700 10 client.14733 handle_command_reply: tid=0 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 client.14733 shutdown 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 msgr2=0x5654448df560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5654448eb180 tx=0x7fae64010640 comp rx=0 tx=0).stop 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 shutdown_connections 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5654449673c0 0x565444968830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654448e63b0 0x5654448dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5654447629a0 0x565444904950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 --2- 172.21.15.50:0/292750205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5654448e0ab0 0x5654448df560 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae57fff700 0 client.14733 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 >> 172.21.15.50:0/292750205 conn(0x5654447c2780 msgr2=0x5654448db110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 shutdown_connections 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 1 -- 172.21.15.50:0/292750205 wait complete. 2024-05-17T04:32:20.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.206+0000 7fae8d674b80 20 client.14733 trim_cache size 0 max 16384 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24506, 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 34.281880262000001, 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 17, 2024-05-17T04:32:20.211 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 87, 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 85, 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 51.535423164999997 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 msgr2=0x7fae680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 0x7fae680b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004630 tx=0x7fae6c00b3c0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 msgr2=0x7fae7819bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b830 tx=0x7fae70004d00 comp rx=0 tx=0).stop 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 shutdown_connections 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fae680b2900 0x7fae680b4e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae781080a0 0x7fae7819bf90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae78102e70 0x7fae7819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 --2- 172.21.15.50:0/1923990132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae78107750 0x7fae7819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 >> 172.21.15.50:0/1923990132 conn(0x7fae78079ae0 msgr2=0x7fae78105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 shutdown_connections 2024-05-17T04:32:20.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:20.207+0000 7fae7ec99700 1 -- 172.21.15.50:0/1923990132 wait complete. 2024-05-17T04:32:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-17T04:32:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-17T04:32:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T04:32:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T04:32:20.316 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:20 smithi105 ceph-mon[34795]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-17T04:32:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-17T04:32:20.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-05-17T04:32:20.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-17T04:32:20.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:20 smithi110 ceph-mon[35497]: osdmap e86: 12 total, 12 up, 12 in 2024-05-17T04:32:20.989 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T04:32:20.989 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:20.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T04:32:21.243 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:21 smithi110 ceph-mon[35497]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T04:32:21.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:21 smithi050 ceph-mon[31739]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T04:32:21.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:21 smithi105 ceph-mon[34795]: osdmap e87: 12 total, 12 up, 12 in 2024-05-17T04:32:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:22 smithi110 ceph-mon[35497]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:22 smithi050 ceph-mon[31739]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:22 smithi105 ceph-mon[34795]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 -- 172.21.15.50:0/3841492035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0102e70 msgr2=0x7f42f0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 --2- 172.21.15.50:0/3841492035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0102e70 0x7f42f0103280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009990 tx=0x7f42e0009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 -- 172.21.15.50:0/3841492035 shutdown_connections 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 --2- 172.21.15.50:0/3841492035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 0x7f42f010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 --2- 172.21.15.50:0/3841492035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0102e70 0x7f42f0103280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 --2- 172.21.15.50:0/3841492035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0107750 0x7f42f0107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 -- 172.21.15.50:0/3841492035 >> 172.21.15.50:0/3841492035 conn(0x7f42f0079ae0 msgr2=0x7f42f0079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:23.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.026+0000 7f42f501b700 1 -- 172.21.15.50:0/3841492035 shutdown_connections 2024-05-17T04:32:23.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.027+0000 7f42f501b700 1 -- 172.21.15.50:0/3841492035 wait complete. 2024-05-17T04:32:23.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 Processor -- start 2024-05-17T04:32:23.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 -- start start 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0102e70 0x7f42f019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 0x7f42f019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0112b80 con 0x7f42f0107750 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0112a00 con 0x7f42f0102e70 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42f501b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0112d00 con 0x7f42f01080a0 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42ee59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42ee59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39620/0 (socket says 172.21.15.50:39620) 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42ee59c700 1 -- 172.21.15.50:0/3510266523 learned_addr learned my addr 172.21.15.50:0/3510266523 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42eed9d700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0102e70 0x7f42f019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.028+0000 7f42ef59e700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 0x7f42f019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:23.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 -- 172.21.15.50:0/3510266523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 msgr2=0x7f42f019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 0x7f42f019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 -- 172.21.15.50:0/3510266523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0102e70 msgr2=0x7f42f019b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0102e70 0x7f42f019b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 -- 172.21.15.50:0/3510266523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f01a6880 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42ee59c700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c8f0 tx=0x7f42e400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e40078d0 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42e400f450 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f01a6b70 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4018660 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.029+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f01a70b0 con 0x7f42f0107750 2024-05-17T04:32:23.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.030+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f0103280 con 0x7f42f0107750 2024-05-17T04:32:23.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.032+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f42e400f900 con 0x7f42f0107750 2024-05-17T04:32:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.033+0000 7f42d7fff700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 0x7f42d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.033+0000 7f42eed9d700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 0x7f42d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:23.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.033+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f42e40e24d0 con 0x7f42f0107750 2024-05-17T04:32:23.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.034+0000 7f42eed9d700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 0x7f42d80b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b5c0 tx=0x7f42e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:23.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.037+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42e4014030 con 0x7f42f0107750 2024-05-17T04:32:23.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.302+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42f006c630 con 0x7f42f0107750 2024-05-17T04:32:23.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.302+0000 7f42d7fff700 1 -- 172.21.15.50:0/3510266523 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f42e40a9ec0 con 0x7f42f0107750 2024-05-17T04:32:23.302 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.304+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 msgr2=0x7f42d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.304+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 0x7f42d80b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b5c0 tx=0x7f42e0005960 comp rx=0 tx=0).stop 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.304+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 msgr2=0x7f42f019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.304+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42e400c8f0 tx=0x7f42e400cc00 comp rx=0 tx=0).stop 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 shutdown_connections 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42f01080a0 0x7f42f019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0102e70 0x7f42f019b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42d80b2880 0x7f42d80b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 --2- 172.21.15.50:0/3510266523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f42f0107750 0x7f42f019ba70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 >> 172.21.15.50:0/3510266523 conn(0x7f42f0079ae0 msgr2=0x7f42f0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 shutdown_connections 2024-05-17T04:32:23.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:23.305+0000 7f42f501b700 1 -- 172.21.15.50:0/3510266523 wait complete. 2024-05-17T04:32:24.121 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-17T04:32:24.121 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T04:32:24.122 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T04:32:24.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:24.122 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:32:24.142 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 42163sec preferred_lft 42163sec 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:32:24.143 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-05-17T04:32:24.144 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T04:32:24.144 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:32:24.144 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:32:24.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-05-17T04:32:24.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-05-17T04:32:24.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-05-17T04:32:24.145 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T04:32:24.145 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:32:24.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:32:24.320 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:24 smithi050 ceph-mon[31739]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:24.320 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:24 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3510266523' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:32:24.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:24.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:24.364 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T04:32:24.390 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-05-17T04:32:24.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:24.391 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-05-17T04:32:24.470 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-05-17T04:32:24.470 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T04:32:24.471 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T04:32:24.471 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T04:32:24.471 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T04:32:24.471 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:32:24.471 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:32:24.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T04:32:24.472 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T04:32:24.472 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T04:32:24.472 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:32:24.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:32:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:24 smithi105 ceph-mon[34795]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:24 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3510266523' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:32:24.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:24 smithi110 ceph-mon[35497]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:24.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:24 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3510266523' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:32:24.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:24.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:24.812 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-17T04:32:24.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:24.829 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-05-17T04:32:24.885 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:32:24.885 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:32:24.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:24.885 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-17T04:32:24.885 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:32:24.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:32:25.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:25.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-17T04:32:25.008 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:32:25.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:32:25.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:25.132 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:32:25.132 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:32:25.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-05-17T04:32:25.132 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T04:32:25.132 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:32:25.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:32:25.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:25.212 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-17T04:32:25.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T04:32:25.212 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:25.228 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-17T04:32:25.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T04:32:25.228 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:25.284 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:32:25.351 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:32:25.408 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:25.409 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:32:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:25.411 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:25.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:25.412 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:32:25.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T04:32:25.466 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-17T04:32:25.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:32:25.534 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:32:25.590 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:32:25.590 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:32:25.590 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:32:25.590 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:32:25.590 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:32:25.591 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:32:25.592 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:25.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:25.593 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:32:25.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:2024-05-17T04:32:25.647+0000 7fbdb3f29480 -1 init, newargv = 0x7fbda40046a0 newargc=15 2024-05-17T04:32:25.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74890]: starting ceph client 2024-05-17T04:32:25.654 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74890]: starting fuse 2024-05-17T04:32:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:26 smithi050 ceph-mon[31739]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:26 smithi105 ceph-mon[34795]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:26 smithi110 ceph-mon[35497]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:26.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:32:26.783 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:32:26.843 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:32:26.844 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:32:26.845 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:32:26.846 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:32:26.846 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:26.846 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:32:26.846 INFO:teuthology.orchestra.run.smithi050.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:26.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:26.847 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:32:26.905 INFO:teuthology.orchestra.run.smithi050.stdout:107 2024-05-17T04:32:26.906 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-17T04:32:26.906 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2024-05-17T04:32:26.906 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2024-05-17T04:32:26.906 DEBUG:teuthology.orchestra.run.smithi050:> import re 2024-05-17T04:32:26.906 DEBUG:teuthology.orchestra.run.smithi050:> import os 2024-05-17T04:32:26.906 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T04:32:26.907 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> return f 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.admin")) 2024-05-17T04:32:26.908 DEBUG:teuthology.orchestra.run.smithi050:> ' 2024-05-17T04:32:27.035 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.admin.74890.asok 2024-05-17T04:32:27.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:27.046 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74890.asok 2024-05-17T04:32:27.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:27.046 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74890.asok status 2024-05-17T04:32:27.195 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:32:27.195 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2024-05-17T04:32:27.195 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "admin", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "74890", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/" 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 14745, 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2024-05-17T04:32:27.196 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 14745 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3532709196 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3532709196 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.14745 192.168.0.1:0/3532709196", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/3532709196", 2024-05-17T04:32:27.197 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 1, 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 17, 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 87, 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2024-05-17T04:32:27.198 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:32:27.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:27.204 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2024-05-17T04:32:27.204 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:27.221 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2024-05-17T04:32:27.221 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:27.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:27.221 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:27.288 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-17T04:32:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-17T04:32:27.405 INFO:teuthology.orchestra.run.smithi050.stdout:# file: . 2024-05-17T04:32:27.405 INFO:teuthology.orchestra.run.smithi050.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-17T04:32:27.405 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:27.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:32:27.411 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:32:27.411 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T04:32:27.428 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T04:32:27.717 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:28.136 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:28 smithi050 ceph-mon[31739]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:28 smithi105 ceph-mon[34795]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:28.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:28 smithi110 ceph-mon[35497]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:29.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- 172.21.15.50:0/3685069712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8103f10 msgr2=0x7fadc8104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:29.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 --2- 172.21.15.50:0/3685069712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8103f10 0x7fadc8104320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009990 tx=0x7fadc4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- 172.21.15.50:0/3685069712 shutdown_connections 2024-05-17T04:32:29.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 --2- 172.21.15.50:0/3685069712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 0x7fadc8110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 --2- 172.21.15.50:0/3685069712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8103f10 0x7fadc8104320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 --2- 172.21.15.50:0/3685069712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8108840 0x7fadc8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- 172.21.15.50:0/3685069712 >> 172.21.15.50:0/3685069712 conn(0x7fadc80ff9b0 msgr2=0x7fadc8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- 172.21.15.50:0/3685069712 shutdown_connections 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- 172.21.15.50:0/3685069712 wait complete. 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 Processor -- start 2024-05-17T04:32:29.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.506+0000 7fadcfacf700 1 -- start start 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8103f10 0x7fadc819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 0x7fadc819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8113c70 con 0x7fadc8108840 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8113af0 con 0x7fadc8109190 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcfacf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8113df0 con 0x7fadc8103f10 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49918/0 (socket says 172.21.15.50:49918) 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 -- 172.21.15.50:0/2103791887 learned_addr learned my addr 172.21.15.50:0/2103791887 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadce06c700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 0x7fadc819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:29.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd86b700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8103f10 0x7fadc819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 -- 172.21.15.50:0/2103791887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8103f10 msgr2=0x7fadc819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8103f10 0x7fadc819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 -- 172.21.15.50:0/2103791887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 msgr2=0x7fadc819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 0x7fadc819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.507+0000 7fadcd06a700 1 -- 172.21.15.50:0/2103791887 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc819f580 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadcd06a700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e970 tx=0x7fadb800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8004a30 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb800cc70 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc819f870 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8005520 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.508+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81a7190 con 0x7fadc8108840 2024-05-17T04:32:29.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.509+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8104320 con 0x7fadc8108840 2024-05-17T04:32:29.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.511+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fadb800c7e0 con 0x7fadc8108840 2024-05-17T04:32:29.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.512+0000 7fadbeffd700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:29.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.512+0000 7fadcd86b700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:29.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.512+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fadb80e2900 con 0x7fadc8108840 2024-05-17T04:32:29.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.512+0000 7fadcd86b700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 0x7fadb40b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b2e0 tx=0x7fadc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:29.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.515+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadb8054df0 con 0x7fadc8108840 2024-05-17T04:32:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.744+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fadc802ea80 con 0x7fadc8108840 2024-05-17T04:32:29.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.744+0000 7fadbeffd700 1 -- 172.21.15.50:0/2103791887 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+589 (secure 0 0 0) 0x7fadb800ce20 con 0x7fadc8108840 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/175302762 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3670657062 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3676423988 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2016476449 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3137083223 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:29.745 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3234552794 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:29.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 msgr2=0x7fadb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:29.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 0x7fadb40b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b2e0 tx=0x7fadc4005fb0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 msgr2=0x7fadc819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:29.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadb800e970 tx=0x7fadb800ec80 comp rx=0 tx=0).stop 2024-05-17T04:32:29.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 shutdown_connections 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc8109190 0x7fadc819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fadc8103f10 0x7fadc819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadb40b2880 0x7fadb40b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 --2- 172.21.15.50:0/2103791887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc8108840 0x7fadc819ac50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 >> 172.21.15.50:0/2103791887 conn(0x7fadc80ff9b0 msgr2=0x7fadc8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 shutdown_connections 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:29.747+0000 7fadcfacf700 1 -- 172.21.15.50:0/2103791887 wait complete. 2024-05-17T04:32:29.748 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-05-17T04:32:30.369 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:30 smithi110 ceph-mon[35497]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:30.370 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:30 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2103791887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:30.467 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:30 smithi050 ceph-mon[31739]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:30.468 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:30 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2103791887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:30.497 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:32:30.497 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T04:32:30.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T04:32:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:30 smithi105 ceph-mon[34795]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:30 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2103791887' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:30.806 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:32.302 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:32 smithi050 ceph-mon[31739]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 384 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:32.367 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:32 smithi105 ceph-mon[34795]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 384 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 -- 172.21.15.50:0/2297875993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38107750 msgr2=0x7f7a38107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 --2- 172.21.15.50:0/2297875993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38107750 0x7f7a38107b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009990 tx=0x7f7a2c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 -- 172.21.15.50:0/2297875993 shutdown_connections 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 --2- 172.21.15.50:0/2297875993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381080a0 0x7f7a3810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 --2- 172.21.15.50:0/2297875993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38102e70 0x7f7a38103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 --2- 172.21.15.50:0/2297875993 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a38107750 0x7f7a38107b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 -- 172.21.15.50:0/2297875993 >> 172.21.15.50:0/2297875993 conn(0x7f7a38079ae0 msgr2=0x7f7a38079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:32.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 -- 172.21.15.50:0/2297875993 shutdown_connections 2024-05-17T04:32:32.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.561+0000 7f7a40167700 1 -- 172.21.15.50:0/2297875993 wait complete. 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 Processor -- start 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 -- start start 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102e70 0x7f7a381995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38107750 0x7f7a38199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112b60 con 0x7f7a381080a0 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381129e0 con 0x7f7a38102e70 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a40167700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112ce0 con 0x7f7a38107750 2024-05-17T04:32:32.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a3e704700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a3e704700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49930/0 (socket says 172.21.15.50:49930) 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a3e704700 1 -- 172.21.15.50:0/2960698618 learned_addr learned my addr 172.21.15.50:0/2960698618 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a3df03700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102e70 0x7f7a381995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.562+0000 7f7a3d702700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38107750 0x7f7a38199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 -- 172.21.15.50:0/2960698618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38107750 msgr2=0x7f7a38199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38107750 0x7f7a38199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 -- 172.21.15.50:0/2960698618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102e70 msgr2=0x7f7a381995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102e70 0x7f7a381995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 -- 172.21.15.50:0/2960698618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3819e400 con 0x7f7a381080a0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a3e704700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c900 tx=0x7f7a3000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30007b00 con 0x7f7a381080a0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a3000ce50 con 0x7f7a381080a0 2024-05-17T04:32:32.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3819e6f0 con 0x7f7a381080a0 2024-05-17T04:32:32.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30018660 con 0x7f7a381080a0 2024-05-17T04:32:32.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.563+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3819ebe0 con 0x7f7a381080a0 2024-05-17T04:32:32.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.564+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a38103280 con 0x7f7a381080a0 2024-05-17T04:32:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.566+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7a3000f450 con 0x7f7a381080a0 2024-05-17T04:32:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.567+0000 7f7a36ffd700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 0x7f7a200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.567+0000 7f7a3df03700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 0x7f7a200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:32.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.567+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7a300e2450 con 0x7f7a381080a0 2024-05-17T04:32:32.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:32 smithi110 ceph-mon[35497]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 384 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-05-17T04:32:32.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.567+0000 7f7a3df03700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 0x7f7a200b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a280060b0 tx=0x7f7a2800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:32.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.571+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a30014030 con 0x7f7a381080a0 2024-05-17T04:32:32.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.802+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a3819a940 con 0x7f7a381080a0 2024-05-17T04:32:32.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.802+0000 7f7a36ffd700 1 -- 172.21.15.50:0/2960698618 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+589 (secure 0 0 0) 0x7f7a300a9e10 con 0x7f7a381080a0 2024-05-17T04:32:32.802 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:32.802 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/175302762 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:32.802 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3670657062 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3676423988 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2016476449 2024-05-18T04:24:18.593378+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3137083223 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:32.803 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3234552794 2024-05-18T04:23:51.164359+0000 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 msgr2=0x7f7a200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 0x7f7a200b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a280060b0 tx=0x7f7a2800c040 comp rx=0 tx=0).stop 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 msgr2=0x7f7a3819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c900 tx=0x7f7a3000cc10 comp rx=0 tx=0).stop 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 shutdown_connections 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7a200b2880 0x7f7a200b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a381080a0 0x7f7a3819de10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102e70 0x7f7a381995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 --2- 172.21.15.50:0/2960698618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a38107750 0x7f7a38199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 >> 172.21.15.50:0/2960698618 conn(0x7f7a38079ae0 msgr2=0x7f7a381054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 shutdown_connections 2024-05-17T04:32:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:32.805+0000 7f7a40167700 1 -- 172.21.15.50:0/2960698618 wait complete. 2024-05-17T04:32:32.806 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-05-17T04:32:33.520 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:33 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2960698618' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:33.552 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2024-05-17T04:32:33.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:33.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-17T04:32:33.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:33 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2960698618' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:33.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:33 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2960698618' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:32:33.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi050.stderr:ceph-fuse[74890]: fuse finished with error 0 and tester_r 0 2024-05-17T04:32:33.582 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T04:32:33.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:32:33.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:32:33.624 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-17T04:32:33.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:32:33.987 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:34.364 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:34 smithi050 ceph-mon[31739]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T04:32:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:34 smithi105 ceph-mon[34795]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T04:32:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:34 smithi110 ceph-mon[35497]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 1 B/s, 0 objects/s recovering 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 -- 172.21.15.50:0/3218087883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c103f20 msgr2=0x7f834c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 --2- 172.21.15.50:0/3218087883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c103f20 0x7f834c104330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f833c009990 tx=0x7f833c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 -- 172.21.15.50:0/3218087883 shutdown_connections 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 --2- 172.21.15.50:0/3218087883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c1091a0 0x7f834c110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 --2- 172.21.15.50:0/3218087883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c103f20 0x7f834c104330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 --2- 172.21.15.50:0/3218087883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c108850 0x7f834c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 -- 172.21.15.50:0/3218087883 >> 172.21.15.50:0/3218087883 conn(0x7f834c0ff9c0 msgr2=0x7f834c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 -- 172.21.15.50:0/3218087883 shutdown_connections 2024-05-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.771+0000 7f835273d700 1 -- 172.21.15.50:0/3218087883 wait complete. 2024-05-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 Processor -- start 2024-05-17T04:32:35.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 -- start start 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 0x7f834c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c108850 0x7f834c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f834c113c80 con 0x7f834c1091a0 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f834c113b00 con 0x7f834c103f20 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.772+0000 7f835273d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f834c113e00 con 0x7f834c108850 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49944/0 (socket says 172.21.15.50:49944) 2024-05-17T04:32:35.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 -- 172.21.15.50:0/4249618084 learned_addr learned my addr 172.21.15.50:0/4249618084 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f834bfff700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 0x7f834c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8343fff700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c108850 0x7f834c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 -- 172.21.15.50:0/4249618084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c108850 msgr2=0x7f834c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c108850 0x7f834c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 -- 172.21.15.50:0/4249618084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 msgr2=0x7f834c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 0x7f834c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 -- 172.21.15.50:0/4249618084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c19f500 con 0x7f834c1091a0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f834bfff700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 0x7f834c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.773+0000 7f8350cda700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8344007ee0 tx=0x7f834400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.774+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834400dbd0 con 0x7f834c1091a0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.774+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f834400dd70 con 0x7f834c1091a0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.774+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83440058d0 con 0x7f834c1091a0 2024-05-17T04:32:35.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.774+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834c19f7f0 con 0x7f834c1091a0 2024-05-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.774+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834c19fce0 con 0x7f834c1091a0 2024-05-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.775+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834c104330 con 0x7f834c1091a0 2024-05-17T04:32:35.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.777+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f834401a020 con 0x7f834c1091a0 2024-05-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.778+0000 7f8349ffb700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 0x7f83380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.778+0000 7f834bfff700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 0x7f83380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.778+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f83440e1c40 con 0x7f834c1091a0 2024-05-17T04:32:35.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.778+0000 7f834bfff700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 0x7f83380b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f833c006010 tx=0x7f833c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:35.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:35.782+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8344010070 con 0x7f834c1091a0 2024-05-17T04:32:36.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.007+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f834c19ba40 con 0x7f834c1091a0 2024-05-17T04:32:36.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.009+0000 7f8349ffb700 1 -- 172.21.15.50:0/4249618084 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24630 (secure 0 0 0) 0x7f83440cb070 con 0x7f834c1091a0 2024-05-17T04:32:36.009 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:36.009 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":87,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:20.048458+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:32:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 msgr2=0x7f83380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 0x7f83380b4e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f833c006010 tx=0x7f833c01a040 comp rx=0 tx=0).stop 2024-05-17T04:32:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 msgr2=0x7f834c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:36.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8344007ee0 tx=0x7f834400d3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 shutdown_connections 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f83380b2900 0x7f83380b4e10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f834c1091a0 0x7f834c19ef10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f834c103f20 0x7f834c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 --2- 172.21.15.50:0/4249618084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f834c108850 0x7f834c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 >> 172.21.15.50:0/4249618084 conn(0x7f834c0ff9c0 msgr2=0x7f834c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 shutdown_connections 2024-05-17T04:32:36.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:36.011+0000 7f835273d700 1 -- 172.21.15.50:0/4249618084 wait complete. 2024-05-17T04:32:36.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:36 smithi050 ceph-mon[31739]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:36.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:36 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/4249618084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:36 smithi105 ceph-mon[34795]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:36 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/4249618084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:36 smithi110 ceph-mon[35497]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:36 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/4249618084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:36.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:32:36.974 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:38.432 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:38 smithi050 ceph-mon[31739]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:38.433 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:32:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:38 smithi105 ceph-mon[34795]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:32:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:38 smithi110 ceph-mon[35497]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 -- 172.21.15.50:0/494939566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 msgr2=0x7f9f0010f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 --2- 172.21.15.50:0/494939566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0010f170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800b0a0 tx=0x7f9ef800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 -- 172.21.15.50:0/494939566 shutdown_connections 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 --2- 172.21.15.50:0/494939566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0010f170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 --2- 172.21.15.50:0/494939566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00102590 0x7f9f001029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 --2- 172.21.15.50:0/494939566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00106ec0 0x7f9f001072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 -- 172.21.15.50:0/494939566 >> 172.21.15.50:0/494939566 conn(0x7f9f00078040 msgr2=0x7f9f000784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 -- 172.21.15.50:0/494939566 shutdown_connections 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 -- 172.21.15.50:0/494939566 wait complete. 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.835+0000 7f9f05306700 1 Processor -- start 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 -- start start 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 0x7f9f0019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00106ec0 0x7f9f0019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111aa0 con 0x7f9f00107810 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111920 con 0x7f9f00106ec0 2024-05-17T04:32:38.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9f05306700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111c20 con 0x7f9f00102590 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9eff7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9eff7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52006/0 (socket says 172.21.15.50:52006) 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9efeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 0x7f9f0019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9eff7fe700 1 -- 172.21.15.50:0/1456953176 learned_addr learned my addr 172.21.15.50:0/1456953176 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.836+0000 7f9efe7fc700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00106ec0 0x7f9f0019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 -- 172.21.15.50:0/1456953176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00106ec0 msgr2=0x7f9f0019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00106ec0 0x7f9f0019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 -- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 msgr2=0x7f9f0019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:38.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 -- 172.21.15.50:0/1456953176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0019f580 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9eff7fe700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9efeffd700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 0x7f9f0019a6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000d750 tx=0x7f9ef000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0014030 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ef00044b0 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0019f810 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0005260 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.837+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f001a7190 con 0x7f9f00102590 2024-05-17T04:32:38.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.838+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f001029a0 con 0x7f9f00102590 2024-05-17T04:32:38.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.840+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9ef00049e0 con 0x7f9f00102590 2024-05-17T04:32:38.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.841+0000 7f9ee7fff700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 0x7f9ee80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:38.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.841+0000 7f9efe7fc700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 0x7f9ee80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:38.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.841+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9ef00e2500 con 0x7f9f00102590 2024-05-17T04:32:38.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.842+0000 7f9efe7fc700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 0x7f9ee80b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f00104020 tx=0x7f9ef4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:38.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:38.844+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ef00a9e60 con 0x7f9f00102590 2024-05-17T04:32:39.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.072+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f0019ba10 con 0x7f9f00102590 2024-05-17T04:32:39.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.074+0000 7f9ee7fff700 1 -- 172.21.15.50:0/1456953176 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24630 (secure 0 0 0) 0x7f9ef00a9760 con 0x7f9f00102590 2024-05-17T04:32:39.074 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:32:39.074 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":87,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:20.048458+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:32:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 msgr2=0x7f9ee80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 0x7f9ee80b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f00104020 tx=0x7f9ef4010040 comp rx=0 tx=0).stop 2024-05-17T04:32:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 msgr2=0x7f9f0019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:39.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 0x7f9f0019a6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000d750 tx=0x7f9ef000da60 comp rx=0 tx=0).stop 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 shutdown_connections 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9ee80b2880 0x7f9ee80b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9f00107810 0x7f9f0019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9f00102590 0x7f9f0019a6b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 --2- 172.21.15.50:0/1456953176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f00106ec0 0x7f9f0019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.076+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 >> 172.21.15.50:0/1456953176 conn(0x7f9f00078040 msgr2=0x7f9f00104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.077+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 shutdown_connections 2024-05-17T04:32:39.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:39.077+0000 7f9f05306700 1 -- 172.21.15.50:0/1456953176 wait complete. 2024-05-17T04:32:39.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:39 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1456953176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:39.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:39 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1456953176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:39.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:39 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1456953176' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:32:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:40 smithi050 ceph-mon[31739]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:40 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-17T04:32:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:40 smithi105 ceph-mon[34795]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:40.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:40 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:40 smithi110 ceph-mon[35497]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:40 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:40.647 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:42.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:42 smithi050 ceph-mon[31739]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:42.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:42 smithi105 ceph-mon[34795]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:42.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:42 smithi110 ceph-mon[35497]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-17T04:32:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 -- 172.21.15.50:0/3995820759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c102e70 msgr2=0x7f781c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 --2- 172.21.15.50:0/3995820759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c102e70 0x7f781c103280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7810009990 tx=0x7f7810009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 -- 172.21.15.50:0/3995820759 shutdown_connections 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 --2- 172.21.15.50:0/3995820759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 --2- 172.21.15.50:0/3995820759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c102e70 0x7f781c103280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 --2- 172.21.15.50:0/3995820759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c107750 0x7f781c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 -- 172.21.15.50:0/3995820759 >> 172.21.15.50:0/3995820759 conn(0x7f781c079ae0 msgr2=0x7f781c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 -- 172.21.15.50:0/3995820759 shutdown_connections 2024-05-17T04:32:42.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.411+0000 7f7822dfb700 1 -- 172.21.15.50:0/3995820759 wait complete. 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.412+0000 7f7822dfb700 1 Processor -- start 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.412+0000 7f7822dfb700 1 -- start start 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 0x7f781c070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f781c112b80 con 0x7f781c107750 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f781c112a00 con 0x7f781c102e70 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7822dfb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f781c112d00 con 0x7f781c1080a0 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52026/0 (socket says 172.21.15.50:52026) 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 -- 172.21.15.50:0/1559370937 learned_addr learned my addr 172.21.15.50:0/1559370937 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7821398700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:42.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7820b97700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 0x7f781c070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 -- 172.21.15.50:0/1559370937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 msgr2=0x7f781c0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c0711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 -- 172.21.15.50:0/1559370937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 msgr2=0x7f781c070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 0x7f781c070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f781bfff700 1 -- 172.21.15.50:0/1559370937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c1af090 con 0x7f781c107750 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7821398700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.413+0000 7f7820b97700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 0x7f781c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f781bfff700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f780800e970 tx=0x7f780800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808004a30 con 0x7f781c107750 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f780800cc70 con 0x7f781c107750 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c06e5a0 con 0x7f781c107750 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808005450 con 0x7f781c107750 2024-05-17T04:32:42.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.414+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c06eb30 con 0x7f781c107750 2024-05-17T04:32:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.415+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c1042d0 con 0x7f781c107750 2024-05-17T04:32:42.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.416+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f78080055f0 con 0x7f781c107750 2024-05-17T04:32:42.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.418+0000 7f7819ffb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 0x7f780c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:42.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.418+0000 7f7820b97700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 0x7f780c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:42.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.418+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f78080e2660 con 0x7f781c107750 2024-05-17T04:32:42.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.418+0000 7f7820b97700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 0x7f780c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f781000b2e0 tx=0x7f7810005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:42.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.422+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f780800ce20 con 0x7f781c107750 2024-05-17T04:32:42.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:42.699+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f781c06c630 con 0x7f781c107750 2024-05-17T04:32:43.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:32:43.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:32:43.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:43.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:43 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T04:32:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:32:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:32:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:43 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T04:32:43.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:32:43.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:32:43.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:32:43.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:43 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-17T04:32:43.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.596+0000 7f7819ffb700 1 -- 172.21.15.50:0/1559370937 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f78080aa020 con 0x7f781c107750 2024-05-17T04:32:43.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 msgr2=0x7f780c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 0x7f780c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f781000b2e0 tx=0x7f7810005fb0 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 msgr2=0x7f781c070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f780800e970 tx=0x7f780800ec80 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 shutdown_connections 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781c1080a0 0x7f781c0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f781c102e70 0x7f781c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f780c0b2900 0x7f780c0b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 --2- 172.21.15.50:0/1559370937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f781c107750 0x7f781c070c90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:43.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.599+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 >> 172.21.15.50:0/1559370937 conn(0x7f781c079ae0 msgr2=0x7f781c1055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:43.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.600+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 shutdown_connections 2024-05-17T04:32:43.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:43.600+0000 7f7822dfb700 1 -- 172.21.15.50:0/1559370937 wait complete. 2024-05-17T04:32:44.298 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:44 smithi050 ceph-mon[31739]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:44.298 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:44 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T04:32:44.298 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:44 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:44 smithi105 ceph-mon[34795]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:44 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T04:32:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:44 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:44 smithi110 ceph-mon[35497]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:44 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1559370937' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-17T04:32:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:44 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:44.326 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-17T04:32:44.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-17T04:32:44.601 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:46.181 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:46 smithi050 ceph-mon[31739]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:46 smithi105 ceph-mon[34795]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:46 smithi110 ceph-mon[35497]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-17T04:32:46.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 -- 172.21.15.50:0/1244422094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4768102e70 msgr2=0x7f4768103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:46.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 --2- 172.21.15.50:0/1244422094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4768102e70 0x7f4768103280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4764009990 tx=0x7f4764009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 -- 172.21.15.50:0/1244422094 shutdown_connections 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 --2- 172.21.15.50:0/1244422094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47681080a0 0x7f476810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 --2- 172.21.15.50:0/1244422094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4768102e70 0x7f4768103280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 --2- 172.21.15.50:0/1244422094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 -- 172.21.15.50:0/1244422094 >> 172.21.15.50:0/1244422094 conn(0x7f4768079ae0 msgr2=0x7f4768079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.456+0000 7f476f75b700 1 -- 172.21.15.50:0/1244422094 shutdown_connections 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.457+0000 7f476f75b700 1 -- 172.21.15.50:0/1244422094 wait complete. 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.457+0000 7f476f75b700 1 Processor -- start 2024-05-17T04:32:46.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.457+0000 7f476f75b700 1 -- start start 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4768102e70 0x7f4768199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112b80 con 0x7f47681080a0 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112a00 con 0x7f4768102e70 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476f75b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112d00 con 0x7f4768107750 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476dcf8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:46.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476ccf6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476dcf8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52040/0 (socket says 172.21.15.50:52040) 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476ccf6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:40970/0 (socket says 172.21.15.50:40970) 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476dcf8700 1 -- 172.21.15.50:0/1037320565 learned_addr learned my addr 172.21.15.50:0/1037320565 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476d4f7700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4768102e70 0x7f4768199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476dcf8700 1 -- 172.21.15.50:0/1037320565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 msgr2=0x7f4768199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.458+0000 7f476dcf8700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476dcf8700 1 -- 172.21.15.50:0/1037320565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4768102e70 msgr2=0x7f4768199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476dcf8700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4768102e70 0x7f4768199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476dcf8700 1 -- 172.21.15.50:0/1037320565 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476819e4d0 con 0x7f47681080a0 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476dcf8700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4760007ee0 tx=0x7f476000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760004bb0 con 0x7f47681080a0 2024-05-17T04:32:46.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4760004d50 con 0x7f47681080a0 2024-05-17T04:32:46.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600058b0 con 0x7f47681080a0 2024-05-17T04:32:46.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476819e7c0 con 0x7f47681080a0 2024-05-17T04:32:46.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.459+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476806c040 con 0x7f47681080a0 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.460+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47681042d0 con 0x7f47681080a0 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.462+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f476001a020 con 0x7f47681080a0 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.463+0000 7f475e7fc700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 0x7f47540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.463+0000 7f476d4f7700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 0x7f47540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.463+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f47600e1c30 con 0x7f47681080a0 2024-05-17T04:32:46.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.463+0000 7f476d4f7700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 0x7f47540b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476400b5c0 tx=0x7f4764005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:46.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.467+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4760010070 con 0x7f47681080a0 2024-05-17T04:32:46.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:46.731+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f476810ae60 con 0x7f47681080a0 2024-05-17T04:32:47.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.018+0000 7f475e7fc700 1 -- 172.21.15.50:0/1037320565 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f47600a9590 con 0x7f47681080a0 2024-05-17T04:32:47.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 msgr2=0x7f47540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:47.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 0x7f47540b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f476400b5c0 tx=0x7f4764005960 comp rx=0 tx=0).stop 2024-05-17T04:32:47.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 msgr2=0x7f476819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:47.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4760007ee0 tx=0x7f476000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:47.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 shutdown_connections 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f47540b2880 0x7f47540b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47681080a0 0x7f476819de30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4768102e70 0x7f4768199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 --2- 172.21.15.50:0/1037320565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4768107750 0x7f4768199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 >> 172.21.15.50:0/1037320565 conn(0x7f4768079ae0 msgr2=0x7f4768106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 shutdown_connections 2024-05-17T04:32:47.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:47.021+0000 7f476f75b700 1 -- 172.21.15.50:0/1037320565 wait complete. 2024-05-17T04:32:47.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:47 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T04:32:47.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:47 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T04:32:47.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:47 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-17T04:32:47.760 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-17T04:32:48.021 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T04:32:48.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:48 smithi105 ceph-mon[34795]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T04:32:48.316 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:48 smithi050 ceph-mon[31739]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1037320565' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-17T04:32:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:48 smithi110 ceph-mon[35497]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-17T04:32:49.199 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:active 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: Cluster is now healthy 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-17T04:32:49.200 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:49 smithi050 ceph-mon[31739]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: 2024-05-17T04:32:49.068+0000 7f1a9f2bb700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: 2024-05-17T04:32:49.068+0000 7f1a9f2bb700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:active 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: Cluster is now healthy 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-17T04:32:49.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:49 smithi105 ceph-mon[34795]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] up:active 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: Cluster is now healthy 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-17T04:32:49.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:49 smithi110 ceph-mon[35497]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 -- 172.21.15.50:0/1263539605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 msgr2=0x7f4d0010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 --2- 172.21.15.50:0/1263539605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0010fa00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b0a0 tx=0x7f4cf800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 -- 172.21.15.50:0/1263539605 shutdown_connections 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 --2- 172.21.15.50:0/1263539605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0010fa00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 --2- 172.21.15.50:0/1263539605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00102e20 0x7f4d00103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 --2- 172.21.15.50:0/1263539605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00107750 0x7f4d00107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 -- 172.21.15.50:0/1263539605 >> 172.21.15.50:0/1263539605 conn(0x7f4d00079af0 msgr2=0x7f4d00079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 -- 172.21.15.50:0/1263539605 shutdown_connections 2024-05-17T04:32:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.750+0000 7f4d04a88700 1 -- 172.21.15.50:0/1263539605 wait complete. 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.751+0000 7f4d04a88700 1 Processor -- start 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 -- start start 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00102e20 0x7f4d0019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00107750 0x7f4d0019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:49.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00112b90 con 0x7f4d001080a0 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00112a10 con 0x7f4d00107750 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4d04a88700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00112d10 con 0x7f4d00102e20 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59300/0 (socket says 172.21.15.50:59300) 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 -- 172.21.15.50:0/161907495 learned_addr learned my addr 172.21.15.50:0/161907495 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfdd9b700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00107750 0x7f4d0019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfe59c700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00102e20 0x7f4d0019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 -- 172.21.15.50:0/161907495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00102e20 msgr2=0x7f4d0019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00102e20 0x7f4d0019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 -- 172.21.15.50:0/161907495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00107750 msgr2=0x7f4d0019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00107750 0x7f4d0019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:49.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.752+0000 7f4cfed9d700 1 -- 172.21.15.50:0/161907495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0019f5d0 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4cfed9d700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b830 tx=0x7f4cf8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8021070 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0019f860 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a71e0 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cf80075c0 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.753+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801dad0 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.754+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce0005330 con 0x7f4d001080a0 2024-05-17T04:32:49.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.755+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4cf8024070 con 0x7f4d001080a0 2024-05-17T04:32:49.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.756+0000 7f4cef7fe700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 0x7f4ce80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:49.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.756+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f4cf80e34a0 con 0x7f4d001080a0 2024-05-17T04:32:49.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.756+0000 7f4cfe59c700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 0x7f4ce80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:49.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.757+0000 7f4cfe59c700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 0x7f4ce80b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005950 tx=0x7f4cf000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:49.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:49.759+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cf802b030 con 0x7f4d001080a0 2024-05-17T04:32:50.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.025+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f4ce0005a40 con 0x7f4d001080a0 2024-05-17T04:32:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.041+0000 7f4cef7fe700 1 -- 172.21.15.50:0/161907495 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f4cf80aaef0 con 0x7f4d001080a0 2024-05-17T04:32:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 msgr2=0x7f4ce80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 0x7f4ce80b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005950 tx=0x7f4cf000b410 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 msgr2=0x7f4d0019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800b830 tx=0x7f4cf8004d00 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 shutdown_connections 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4ce80b2880 0x7f4ce80b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d001080a0 0x7f4d0019ef30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d00102e20 0x7f4d0019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 --2- 172.21.15.50:0/161907495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d00107750 0x7f4d0019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.043+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 >> 172.21.15.50:0/161907495 conn(0x7f4d00079af0 msgr2=0x7f4d0006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.044+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 shutdown_connections 2024-05-17T04:32:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:50.044+0000 7f4d04a88700 1 -- 172.21.15.50:0/161907495 wait complete. 2024-05-17T04:32:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:50 smithi105 ceph-mon[34795]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-17T04:32:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:50 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:active 2024-05-17T04:32:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:50 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:50.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:50 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T04:32:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:50 smithi110 ceph-mon[35497]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-17T04:32:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:50 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:active 2024-05-17T04:32:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:50 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:50 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T04:32:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:50 smithi050 ceph-mon[31739]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-17T04:32:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:50 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] up:active 2024-05-17T04:32:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:50 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:50 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-17T04:32:50.752 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-17T04:32:51.008 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:51.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:51 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T04:32:51.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:51 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:51.311 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:51 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:51.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:51 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T04:32:51.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:51 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:51.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:51 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:51 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/161907495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-17T04:32:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:51 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-17T04:32:51.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:51 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:52.202 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:52 smithi050 ceph-mon[31739]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-17T04:32:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:52 smithi105 ceph-mon[34795]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-17T04:32:52.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:52 smithi110 ceph-mon[35497]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 -- 172.21.15.50:0/2261098546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c102e00 msgr2=0x7f426c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 --2- 172.21.15.50:0/2261098546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c102e00 0x7f426c103210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4254009990 tx=0x7f4254009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 -- 172.21.15.50:0/2261098546 shutdown_connections 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 --2- 172.21.15.50:0/2261098546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c108080 0x7f426c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 --2- 172.21.15.50:0/2261098546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c102e00 0x7f426c103210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 --2- 172.21.15.50:0/2261098546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 -- 172.21.15.50:0/2261098546 >> 172.21.15.50:0/2261098546 conn(0x7f426c079bd0 msgr2=0x7f426c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 -- 172.21.15.50:0/2261098546 shutdown_connections 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.784+0000 7f4272e65700 1 -- 172.21.15.50:0/2261098546 wait complete. 2024-05-17T04:32:52.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 Processor -- start 2024-05-17T04:32:52.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 -- start start 2024-05-17T04:32:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c102e00 0x7f426c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f426c112b60 con 0x7f426c108080 2024-05-17T04:32:52.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f426c1129e0 con 0x7f426c102e00 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4272e65700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f426c112ce0 con 0x7f426c107730 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4271402700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4271402700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59312/0 (socket says 172.21.15.50:59312) 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4271402700 1 -- 172.21.15.50:0/1884597444 learned_addr learned my addr 172.21.15.50:0/1884597444 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f4270c01700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c102e00 0x7f426c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.785+0000 7f426bfff700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 -- 172.21.15.50:0/1884597444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 msgr2=0x7f426c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 -- 172.21.15.50:0/1884597444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c102e00 msgr2=0x7f426c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c102e00 0x7f426c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 -- 172.21.15.50:0/1884597444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c19f580 con 0x7f426c108080 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f426bfff700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:32:52.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4271402700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f426400e940 tx=0x7f426400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426400cbd0 con 0x7f426c108080 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c19f870 con 0x7f426c108080 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f426400ee90 con 0x7f426c108080 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264005520 con 0x7f426c108080 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.786+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c1a7190 con 0x7f426c108080 2024-05-17T04:32:52.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.787+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c103210 con 0x7f426c108080 2024-05-17T04:32:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.789+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4264004750 con 0x7f426c108080 2024-05-17T04:32:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.789+0000 7f4269ffb700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 0x7f42580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.790+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f42640e2770 con 0x7f426c108080 2024-05-17T04:32:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.790+0000 7f4270c01700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 0x7f42580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:52.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.790+0000 7f4270c01700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 0x7f42580b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f425400b2e0 tx=0x7f4254005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:52.794+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42640aa130 con 0x7f426c108080 2024-05-17T04:32:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:53.057+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f426c1a7780 con 0x7f426c108080 2024-05-17T04:32:54.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.056+0000 7f4269ffb700 1 -- 172.21.15.50:0/1884597444 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f4264005700 con 0x7f426c108080 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.059+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 msgr2=0x7f42580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.059+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 0x7f42580b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f425400b2e0 tx=0x7f4254005fb0 comp rx=0 tx=0).stop 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.059+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 msgr2=0x7f426c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.059+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f426400e940 tx=0x7f426400ec50 comp rx=0 tx=0).stop 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 shutdown_connections 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f42580b2880 0x7f42580b4d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:54.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f426c108080 0x7f426c19eee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:54.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f426c102e00 0x7f426c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:54.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 --2- 172.21.15.50:0/1884597444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f426c107730 0x7f426c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:54.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 >> 172.21.15.50:0/1884597444 conn(0x7f426c079bd0 msgr2=0x7f426c106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:54.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 shutdown_connections 2024-05-17T04:32:54.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:54.060+0000 7f4272e65700 1 -- 172.21.15.50:0/1884597444 wait complete. 2024-05-17T04:32:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:54 smithi105 ceph-mon[34795]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-17T04:32:54.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:54 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T04:32:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:54 smithi110 ceph-mon[35497]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-17T04:32:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:54 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T04:32:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:54 smithi050 ceph-mon[31739]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-17T04:32:54.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:54 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-17T04:32:54.787 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-17T04:32:54.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-17T04:32:55.046 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:55.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:55 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T04:32:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:55 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:55 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T04:32:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:55 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:55.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:55 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1884597444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-17T04:32:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:55 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:32:56.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:56 smithi110 ceph-mon[35497]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:56 smithi050 ceph-mon[31739]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:56 smithi105 ceph-mon[34795]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:56.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2574468726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88108840 msgr2=0x7f4f88108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2574468726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88108840 0x7f4f88108c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009990 tx=0x7f4f70009ca0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2574468726 shutdown_connections 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2574468726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88109190 0x7f4f88110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2574468726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f88104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2574468726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88108840 0x7f4f88108c50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2574468726 >> 172.21.15.50:0/2574468726 conn(0x7f4f880ff9b0 msgr2=0x7f4f88101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2574468726 shutdown_connections 2024-05-17T04:32:56.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.940+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2574468726 wait complete. 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 Processor -- start 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 -- start start 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 0x7f4f881a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 0x7f4f8819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88113c50 con 0x7f4f88109190 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88113ad0 con 0x7f4f88103f10 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.941+0000 7f4f8cdaa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88113dd0 con 0x7f4f88108840 2024-05-17T04:32:56.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f8659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 0x7f4f8819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f7ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 0x7f4f881a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f8659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:50012/0 (socket says 172.21.15.50:50012) 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f7ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 0x7f4f881a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:55544/0 (socket says 172.21.15.50:55544) 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f8659c700 1 -- 172.21.15.50:0/2632119531 learned_addr learned my addr 172.21.15.50:0/2632119531 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 -- 172.21.15.50:0/2632119531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 msgr2=0x7f4f881a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 0x7f4f881a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 -- 172.21.15.50:0/2632119531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 msgr2=0x7f4f881a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 -- 172.21.15.50:0/2632119531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8819ce70 con 0x7f4f88109190 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f8659c700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f86d9d700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 0x7f4f8819c740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007f10 tx=0x7f4f8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.942+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004db0 con 0x7f4f88109190 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.943+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8819d160 con 0x7f4f88109190 2024-05-17T04:32:56.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.943+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f80009e20 con 0x7f4f88109190 2024-05-17T04:32:56.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.943+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80015400 con 0x7f4f88109190 2024-05-17T04:32:56.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.943+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f881a9360 con 0x7f4f88109190 2024-05-17T04:32:56.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.944+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f881053c0 con 0x7f4f88109190 2024-05-17T04:32:56.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.946+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4f80005240 con 0x7f4f88109190 2024-05-17T04:32:56.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.946+0000 7f4f7f7fe700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 0x7f4f740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:32:56.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.947+0000 7f4f8659c700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 0x7f4f740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:32:56.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.947+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f4f800e26c0 con 0x7f4f88109190 2024-05-17T04:32:56.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.947+0000 7f4f8659c700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 0x7f4f740b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f78011550 tx=0x7f4f78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:32:56.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:56.950+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f80010070 con 0x7f4f88109190 2024-05-17T04:32:57.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.196+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4f88076a70 con 0x7f4f740b2880 2024-05-17T04:32:57.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.229+0000 7f4f7f7fe700 1 -- 172.21.15.50:0/2632119531 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f88076a70 con 0x7f4f740b2880 2024-05-17T04:32:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 msgr2=0x7f4f740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 0x7f4f740b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f78011550 tx=0x7f4f78009260 comp rx=0 tx=0).stop 2024-05-17T04:32:57.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 msgr2=0x7f4f8819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 0x7f4f8819c740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007f10 tx=0x7f4f8000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 shutdown_connections 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f740b2880 0x7f4f740b4d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f88109190 0x7f4f8819c740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f88103f10 0x7f4f881a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 --2- 172.21.15.50:0/2632119531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f88108840 0x7f4f881a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 >> 172.21.15.50:0/2632119531 conn(0x7f4f880ff9b0 msgr2=0x7f4f8810a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 shutdown_connections 2024-05-17T04:32:57.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:32:57.231+0000 7f4f8cdaa700 1 -- 172.21.15.50:0/2632119531 wait complete. 2024-05-17T04:32:57.550 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:32:57 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:32:57.202+0000 7fe3e372c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T04:32:57.550 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:32:57 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:32:57.202+0000 7fe3e372c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T04:32:57.550 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:32:57 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:32:57.202+0000 7fe3e372c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T04:32:57.550 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:32:57 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:32:57.202+0000 7fe3e372c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T04:32:57.550 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:32:57 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:32:57.202+0000 7fe3e372c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-17T04:32:57.906 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-17T04:32:58.165 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:32:58.288 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:58 smithi050 ceph-mon[31739]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:58 smithi105 ceph-mon[34795]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:58 smithi110 ceph-mon[35497]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:32:59.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:32:59 smithi050 ceph-mon[31739]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:32:59.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:32:59 smithi105 ceph-mon[34795]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:32:59.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:32:59 smithi110 ceph-mon[35497]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:00.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.099+0000 7faa7c86d700 1 -- 172.21.15.50:0/251537568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103c70 msgr2=0x7faa7810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:00.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.099+0000 7faa7c86d700 1 --2- 172.21.15.50:0/251537568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103c70 0x7faa7810f950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faa7000b0a0 tx=0x7faa7000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 -- 172.21.15.50:0/251537568 shutdown_connections 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 --2- 172.21.15.50:0/251537568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103c70 0x7faa7810f950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 --2- 172.21.15.50:0/251537568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 0x7faa78109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 --2- 172.21.15.50:0/251537568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103320 0x7faa78103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 -- 172.21.15.50:0/251537568 >> 172.21.15.50:0/251537568 conn(0x7faa78078040 msgr2=0x7faa780784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 -- 172.21.15.50:0/251537568 shutdown_connections 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 -- 172.21.15.50:0/251537568 wait complete. 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.100+0000 7faa7c86d700 1 Processor -- start 2024-05-17T04:33:00.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 -- start start 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103c70 0x7faa78104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 0x7faa78105420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112b90 con 0x7faa78103320 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112a10 con 0x7faa78108c10 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7c86d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112d10 con 0x7faa78103c70 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54040/0 (socket says 172.21.15.50:54040) 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa7659c700 1 -- 172.21.15.50:0/2914542825 learned_addr learned my addr 172.21.15.50:0/2914542825 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa75d9b700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103c70 0x7faa78104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.101+0000 7faa76d9d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 0x7faa78105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:00.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 -- 172.21.15.50:0/2914542825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103c70 msgr2=0x7faa78104ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:00.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103c70 0x7faa78104ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 -- 172.21.15.50:0/2914542825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 msgr2=0x7faa78105420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:00.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 0x7faa78105420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 -- 172.21.15.50:0/2914542825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781a8a40 con 0x7faa78103320 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7659c700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa6800d950 tx=0x7faa6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68016070 con 0x7faa78103320 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a8cd0 con 0x7faa78103320 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa6800fd20 con 0x7faa78103320 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6800e3b0 con 0x7faa78103320 2024-05-17T04:33:00.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.102+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a91b0 con 0x7faa78103320 2024-05-17T04:33:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.103+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78103730 con 0x7faa78103320 2024-05-17T04:33:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.105+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faa68019070 con 0x7faa78103320 2024-05-17T04:33:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.106+0000 7faa677fe700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 0x7faa600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.106+0000 7faa75d9b700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 0x7faa600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.106+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7faa680e2110 con 0x7faa78103320 2024-05-17T04:33:00.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.107+0000 7faa75d9b700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 0x7faa600b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa78103260 tx=0x7faa6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.110+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa680a9970 con 0x7faa78103320 2024-05-17T04:33:00.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.359+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faa78100c00 con 0x7faa600b2910 2024-05-17T04:33:00.360 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:00 smithi050 ceph-mon[31739]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:33:00.360 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:00 smithi050 ceph-mon[31739]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T04:33:00.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.388+0000 7faa677fe700 1 -- 172.21.15.50:0/2914542825 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa78100c00 con 0x7faa600b2910 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 msgr2=0x7faa600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 0x7faa600b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa78103260 tx=0x7faa6c00b410 comp rx=0 tx=0).stop 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 msgr2=0x7faa781049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa6800d950 tx=0x7faa6800dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 shutdown_connections 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78108c10 0x7faa78105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7faa600b2910 0x7faa600b4e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa78103320 0x7faa781049a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 --2- 172.21.15.50:0/2914542825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa78103c70 0x7faa78104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 >> 172.21.15.50:0/2914542825 conn(0x7faa78078040 msgr2=0x7faa78106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 shutdown_connections 2024-05-17T04:33:00.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:00.390+0000 7faa7c86d700 1 -- 172.21.15.50:0/2914542825 wait complete. 2024-05-17T04:33:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:00 smithi110 ceph-mon[35497]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:33:00.469 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:00 smithi110 ceph-mon[35497]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T04:33:00.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:00 smithi105 ceph-mon[34795]: pgmap v337: 129 pgs: 129 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-17T04:33:00.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:00 smithi105 ceph-mon[34795]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-17T04:33:01.091 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-17T04:33:01.344 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:02.367 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:02 smithi105 ceph-mon[34795]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:02.367 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:02 smithi105 ceph-mon[34795]: pgmap v338: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-17T04:33:02.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:02 smithi110 ceph-mon[35497]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:02.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:02 smithi110 ceph-mon[35497]: pgmap v338: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-17T04:33:02.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:02 smithi050 ceph-mon[31739]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:02.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:02 smithi050 ceph-mon[31739]: pgmap v338: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 -- 172.21.15.50:0/3438666573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481031d0 msgr2=0x7f264810fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 --2- 172.21.15.50:0/3438666573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481031d0 0x7f264810fe60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2638009990 tx=0x7f2638009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 -- 172.21.15.50:0/3438666573 shutdown_connections 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 --2- 172.21.15.50:0/3438666573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 --2- 172.21.15.50:0/3438666573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481027f0 0x7f2648102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 --2- 172.21.15.50:0/3438666573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481031d0 0x7f264810fe60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 -- 172.21.15.50:0/3438666573 >> 172.21.15.50:0/3438666573 conn(0x7f26480fe6d0 msgr2=0x7f2648100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:03.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 -- 172.21.15.50:0/3438666573 shutdown_connections 2024-05-17T04:33:03.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.210+0000 7f264e7a0700 1 -- 172.21.15.50:0/3438666573 wait complete. 2024-05-17T04:33:03.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.211+0000 7f264e7a0700 1 Processor -- start 2024-05-17T04:33:03.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.211+0000 7f264e7a0700 1 -- start start 2024-05-17T04:33:03.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481031d0 0x7f264819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2648115cf0 con 0x7f26481027f0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2648115b70 con 0x7f26481031d0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2648115e70 con 0x7f26481103a0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54062/0 (socket says 172.21.15.50:54062) 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 -- 172.21.15.50:0/1803908411 learned_addr learned my addr 172.21.15.50:0/1803908411 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264df9f700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264cf9d700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481031d0 0x7f264819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 -- 172.21.15.50:0/1803908411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 msgr2=0x7f26481a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:03.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 -- 172.21.15.50:0/1803908411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481031d0 msgr2=0x7f264819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481031d0 0x7f264819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 -- 172.21.15.50:0/1803908411 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a1530 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264df9f700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264d79e700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f264400c9b0 tx=0x7f264400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644007b00 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2644007ca0 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.212+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a1820 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.213+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644018660 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.213+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a1d60 con 0x7f26481027f0 2024-05-17T04:33:03.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.214+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2648102c00 con 0x7f26481027f0 2024-05-17T04:33:03.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.215+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f264400f450 con 0x7f26481027f0 2024-05-17T04:33:03.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.216+0000 7f263e7fc700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 0x7f26340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:03.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.216+0000 7f264cf9d700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 0x7f26340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:03.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.216+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f26440e27b0 con 0x7f26481027f0 2024-05-17T04:33:03.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.216+0000 7f264cf9d700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 0x7f26340b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f263800b5c0 tx=0x7f2638005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:03.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.220+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2644014030 con 0x7f26481027f0 2024-05-17T04:33:03.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.471+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f264819da70 con 0x7f26340b2910 2024-05-17T04:33:03.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.478+0000 7f263e7fc700 1 -- 172.21.15.50:0/1803908411 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f264819da70 con 0x7f26340b2910 2024-05-17T04:33:03.479 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_0/ddab5ecc-ad15-4ecc-94b9-04078a785401 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 msgr2=0x7f26340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 0x7f26340b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f263800b5c0 tx=0x7f2638005fb0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 msgr2=0x7f264819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f264400c9b0 tx=0x7f264400cd70 comp rx=0 tx=0).stop 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 shutdown_connections 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26481103a0 0x7f26481a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26340b2910 0x7f26340b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26481027f0 0x7f264819c710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 --2- 172.21.15.50:0/1803908411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481031d0 0x7f264819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 >> 172.21.15.50:0/1803908411 conn(0x7f26480fe6d0 msgr2=0x7f2648101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 shutdown_connections 2024-05-17T04:33:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:03.481+0000 7f264e7a0700 1 -- 172.21.15.50:0/1803908411 wait complete. 2024-05-17T04:33:04.163 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:04 smithi050 ceph-mon[31739]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-17T04:33:04.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-17T04:33:04.450 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:04 smithi105 ceph-mon[34795]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-17T04:33:04.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:04 smithi110 ceph-mon[35497]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-17T04:33:05.318 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:05 smithi050 ceph-mon[31739]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:05.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:05 smithi105 ceph-mon[34795]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:05.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:05 smithi110 ceph-mon[35497]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 -- 172.21.15.50:0/172388397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758103f10 msgr2=0x7f9758104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 --2- 172.21.15.50:0/172388397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758103f10 0x7f9758104320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f974800b0a0 tx=0x7f974800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 -- 172.21.15.50:0/172388397 shutdown_connections 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 --2- 172.21.15.50:0/172388397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758109190 0x7f9758110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 --2- 172.21.15.50:0/172388397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758103f10 0x7f9758104320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 --2- 172.21.15.50:0/172388397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758108840 0x7f9758108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 -- 172.21.15.50:0/172388397 >> 172.21.15.50:0/172388397 conn(0x7f97580ff9b0 msgr2=0x7f9758101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 -- 172.21.15.50:0/172388397 shutdown_connections 2024-05-17T04:33:06.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.272+0000 7f975cba2700 1 -- 172.21.15.50:0/172388397 wait complete. 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.273+0000 7f975cba2700 1 Processor -- start 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.273+0000 7f975cba2700 1 -- start start 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758103f10 0x7f9758075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758108840 0x7f9758076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113c70 con 0x7f9758109190 2024-05-17T04:33:06.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113af0 con 0x7f9758108840 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975cba2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9758113df0 con 0x7f9758103f10 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54076/0 (socket says 172.21.15.50:54076) 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 -- 172.21.15.50:0/3081641426 learned_addr learned my addr 172.21.15.50:0/3081641426 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f975659c700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758103f10 0x7f9758075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9755d9b700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758108840 0x7f9758076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 -- 172.21.15.50:0/3081641426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758103f10 msgr2=0x7f9758075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758103f10 0x7f9758075b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 -- 172.21.15.50:0/3081641426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758108840 msgr2=0x7f9758076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758108840 0x7f9758076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 -- 172.21.15.50:0/3081641426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581a8b10 con 0x7f9758109190 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.274+0000 7f9756d9d700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f975000d980 tx=0x7f975000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.275+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750017070 con 0x7f9758109190 2024-05-17T04:33:06.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.275+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581a8da0 con 0x7f9758109190 2024-05-17T04:33:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.275+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9750005020 con 0x7f9758109190 2024-05-17T04:33:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.275+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975000f980 con 0x7f9758109190 2024-05-17T04:33:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.275+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581a9250 con 0x7f9758109190 2024-05-17T04:33:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.276+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9758104320 con 0x7f9758109190 2024-05-17T04:33:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.277+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9750004a30 con 0x7f9758109190 2024-05-17T04:33:06.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.278+0000 7f97477fe700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 0x7f97400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:06.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.278+0000 7f975659c700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 0x7f97400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:06.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.278+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f97500e2880 con 0x7f9758109190 2024-05-17T04:33:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.279+0000 7f975659c700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 0x7f97400b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9748009900 tx=0x7f974800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:06.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.281+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9750020020 con 0x7f9758109190 2024-05-17T04:33:06.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:06 smithi050 ceph-mon[31739]: pgmap v340: 129 pgs: 129 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T04:33:06.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.529+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9758077060 con 0x7f97400b2910 2024-05-17T04:33:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.557+0000 7f97477fe700 1 -- 172.21.15.50:0/3081641426 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9758077060 con 0x7f97400b2910 2024-05-17T04:33:06.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 msgr2=0x7f97400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 0x7f97400b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9748009900 tx=0x7f974800b7b0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 msgr2=0x7f97580765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f975000d980 tx=0x7f975000dc90 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 shutdown_connections 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97400b2910 0x7f97400b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9758109190 0x7f97580765b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9758103f10 0x7f9758075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 --2- 172.21.15.50:0/3081641426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9758108840 0x7f9758076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.559+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 >> 172.21.15.50:0/3081641426 conn(0x7f97580ff9b0 msgr2=0x7f9758100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.560+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 shutdown_connections 2024-05-17T04:33:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:06.560+0000 7f975cba2700 1 -- 172.21.15.50:0/3081641426 wait complete. 2024-05-17T04:33:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:06 smithi105 ceph-mon[34795]: pgmap v340: 129 pgs: 129 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T04:33:06.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:06 smithi110 ceph-mon[35497]: pgmap v340: 129 pgs: 129 active+clean; 610 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-17T04:33:07.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-17T04:33:07.526 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:08 smithi050 ceph-mon[31739]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:08.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:08 smithi050 ceph-mon[31739]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:08 smithi105 ceph-mon[34795]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:08.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:08 smithi105 ceph-mon[34795]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:08 smithi110 ceph-mon[35497]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:08 smithi110 ceph-mon[35497]: pgmap v341: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 -- 172.21.15.50:0/3175761692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4102650 msgr2=0x7f55f4102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 --2- 172.21.15.50:0/3175761692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4102650 0x7f55f4102a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009990 tx=0x7f55e4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 -- 172.21.15.50:0/3175761692 shutdown_connections 2024-05-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 --2- 172.21.15.50:0/3175761692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4110890 0x7f55f4112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 --2- 172.21.15.50:0/3175761692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 0x7f55f4102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 --2- 172.21.15.50:0/3175761692 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4102650 0x7f55f4102a60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.372+0000 7f55fc15e700 1 -- 172.21.15.50:0/3175761692 >> 172.21.15.50:0/3175761692 conn(0x7f55f4078040 msgr2=0x7f55f40784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:09.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.373+0000 7f55fc15e700 1 -- 172.21.15.50:0/3175761692 shutdown_connections 2024-05-17T04:33:09.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.373+0000 7f55fc15e700 1 -- 172.21.15.50:0/3175761692 wait complete. 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.373+0000 7f55fc15e700 1 Processor -- start 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 -- start start 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4102650 0x7f55f41a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 0x7f55f41a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4115e50 con 0x7f55f4110890 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4115cd0 con 0x7f55f4102650 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fc15e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55f4115fd0 con 0x7f55f4104bb0 2024-05-17T04:33:09.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fa6fb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fa6fb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40344/0 (socket says 172.21.15.50:40344) 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55fa6fb700 1 -- 172.21.15.50:0/2500152096 learned_addr learned my addr 172.21.15.50:0/2500152096 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55f9efa700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4102650 0x7f55f41a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.374+0000 7f55f96f9700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 0x7f55f41a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 -- 172.21.15.50:0/2500152096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 msgr2=0x7f55f41a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 0x7f55f41a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 -- 172.21.15.50:0/2500152096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4102650 msgr2=0x7f55f41a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4102650 0x7f55f41a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 -- 172.21.15.50:0/2500152096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f419ce70 con 0x7f55f4110890 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fa6fb700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d980 tx=0x7f55ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec017070 con 0x7f55f4110890 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55ec005020 con 0x7f55f4110890 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00f980 con 0x7f55f4110890 2024-05-17T04:33:09.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f419d160 con 0x7f55f4110890 2024-05-17T04:33:09.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.375+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f41a9360 con 0x7f55f4110890 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.376+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f4103330 con 0x7f55f4110890 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.378+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f55ec004a30 con 0x7f55f4110890 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.378+0000 7f55eaffd700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 0x7f55e00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.379+0000 7f55f9efa700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 0x7f55e00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.379+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f55ec0e2460 con 0x7f55f4110890 2024-05-17T04:33:09.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.379+0000 7f55f9efa700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 0x7f55e00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55f00060b0 tx=0x7f55f000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:09.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.383+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55ec020020 con 0x7f55f4110890 2024-05-17T04:33:09.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.633+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f55f4112280 con 0x7f55e00b2910 2024-05-17T04:33:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.638+0000 7f55eaffd700 1 -- 172.21.15.50:0/2500152096 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f55f4112280 con 0x7f55e00b2910 2024-05-17T04:33:09.638 INFO:teuthology.orchestra.run.smithi050.stdout:/volumes/qa/sv_1/52398635-e58b-4000-b46c-86c7ff0f627e 2024-05-17T04:33:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 msgr2=0x7f55e00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 0x7f55e00b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55f00060b0 tx=0x7f55f000c040 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 msgr2=0x7f55f419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d980 tx=0x7f55ec00dc90 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 shutdown_connections 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f55e00b2910 0x7f55e00b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55f4110890 0x7f55f419c740 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f4102650 0x7f55f41a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 --2- 172.21.15.50:0/2500152096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55f4104bb0 0x7f55f41a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.640+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 >> 172.21.15.50:0/2500152096 conn(0x7f55f4078040 msgr2=0x7f55f4111b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.641+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 shutdown_connections 2024-05-17T04:33:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:09.641+0000 7f55fc15e700 1 -- 172.21.15.50:0/2500152096 wait complete. 2024-05-17T04:33:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:10 smithi050 ceph-mon[31739]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-17T04:33:10.458 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:33:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:10 smithi105 ceph-mon[34795]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-17T04:33:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:10 smithi110 ceph-mon[35497]: pgmap v342: 129 pgs: 129 active+clean; 611 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-17T04:33:10.716 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:11.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:11 smithi105 ceph-mon[34795]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:11 smithi110 ceph-mon[35497]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:11.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:11 smithi050 ceph-mon[31739]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:33:12.155 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:12 smithi050 ceph-mon[31739]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T04:33:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:12 smithi105 ceph-mon[34795]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T04:33:12.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 -- 172.21.15.50:0/3267747634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 msgr2=0x7fd178108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:12.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 --2- 172.21.15.50:0/3267747634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd178108c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009990 tx=0x7fd16c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 -- 172.21.15.50:0/3267747634 shutdown_connections 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 --2- 172.21.15.50:0/3267747634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 0x7fd178110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 --2- 172.21.15.50:0/3267747634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 0x7fd178104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 --2- 172.21.15.50:0/3267747634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd178108c60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 -- 172.21.15.50:0/3267747634 >> 172.21.15.50:0/3267747634 conn(0x7fd1780ff9c0 msgr2=0x7fd178101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 -- 172.21.15.50:0/3267747634 shutdown_connections 2024-05-17T04:33:12.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.566+0000 7fd17cb89700 1 -- 172.21.15.50:0/3267747634 wait complete. 2024-05-17T04:33:12.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:12 smithi110 ceph-mon[35497]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-05-17T04:33:12.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.567+0000 7fd17cb89700 1 Processor -- start 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.567+0000 7fd17cb89700 1 -- start start 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 0x7fd1781a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 0x7fd17819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd178113d30 con 0x7fd178108850 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd178113bb0 con 0x7fd1781091a0 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17cb89700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd178113eb0 con 0x7fd178103f20 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40368/0 (socket says 172.21.15.50:40368) 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 -- 172.21.15.50:0/1612636443 learned_addr learned my addr 172.21.15.50:0/1612636443 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:12.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd176d9d700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 0x7fd17819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd17659c700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 0x7fd1781a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 -- 172.21.15.50:0/1612636443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 msgr2=0x7fd1781a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 0x7fd1781a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 -- 172.21.15.50:0/1612636443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 msgr2=0x7fd17819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 0x7fd17819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 -- 172.21.15.50:0/1612636443 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819ad80 con 0x7fd178108850 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd175d9b700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.568+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c027070 con 0x7fd178108850 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.569+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd16c022470 con 0x7fd178108850 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.569+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819b010 con 0x7fd178108850 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.569+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c017650 con 0x7fd178108850 2024-05-17T04:33:12.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.569+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a72a0 con 0x7fd178108850 2024-05-17T04:33:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.569+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17810c230 con 0x7fd178108850 2024-05-17T04:33:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.571+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd16c02a070 con 0x7fd178108850 2024-05-17T04:33:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.572+0000 7fd1677fe700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 0x7fd1600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:12.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.572+0000 7fd17659c700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 0x7fd1600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.572+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd16c0e37d0 con 0x7fd178108850 2024-05-17T04:33:12.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.572+0000 7fd17659c700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 0x7fd1600b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd17819baf0 tx=0x7fd16800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:12.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.575+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd16c013070 con 0x7fd178108850 2024-05-17T04:33:12.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.838+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd178104330 con 0x7fd178108850 2024-05-17T04:33:12.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.839+0000 7fd1677fe700 1 -- 172.21.15.50:0/1612636443 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7fd16c0ab170 con 0x7fd178108850 2024-05-17T04:33:12.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:12.840 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:33:12.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 msgr2=0x7fd1600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:12.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 0x7fd1600b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd17819baf0 tx=0x7fd16800d040 comp rx=0 tx=0).stop 2024-05-17T04:33:12.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 msgr2=0x7fd1781a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd16c000c00 tx=0x7fd16c0057c0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 shutdown_connections 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1781091a0 0x7fd17819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd178103f20 0x7fd1781a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd1600b2990 0x7fd1600b4ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 --2- 172.21.15.50:0/1612636443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd178108850 0x7fd1781a09e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.841+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 >> 172.21.15.50:0/1612636443 conn(0x7fd1780ff9c0 msgr2=0x7fd178101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.842+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 shutdown_connections 2024-05-17T04:33:12.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:12.842+0000 7fd17cb89700 1 -- 172.21.15.50:0/1612636443 wait complete. 2024-05-17T04:33:12.843 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:33:13.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:13 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1612636443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:33:13.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:13 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1612636443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:33:13.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:13 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1612636443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:33:13.720 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:33:13.989 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:14.307 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:14 smithi050 ceph-mon[31739]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T04:33:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:14 smithi105 ceph-mon[34795]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T04:33:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:14 smithi110 ceph-mon[35497]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-05-17T04:33:15.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 -- 172.21.15.50:0/1679340239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 msgr2=0x7f61d0107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:15.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 --2- 172.21.15.50:0/1679340239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0107b60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009990 tx=0x7f61c4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 -- 172.21.15.50:0/1679340239 shutdown_connections 2024-05-17T04:33:15.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 --2- 172.21.15.50:0/1679340239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d01080a0 0x7f61d010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 --2- 172.21.15.50:0/1679340239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d0102e20 0x7f61d0103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 --2- 172.21.15.50:0/1679340239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0107b60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 -- 172.21.15.50:0/1679340239 >> 172.21.15.50:0/1679340239 conn(0x7f61d0079bd0 msgr2=0x7f61d0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 -- 172.21.15.50:0/1679340239 shutdown_connections 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.908+0000 7f61d6588700 1 -- 172.21.15.50:0/1679340239 wait complete. 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.909+0000 7f61d6588700 1 Processor -- start 2024-05-17T04:33:15.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.909+0000 7f61d6588700 1 -- start start 2024-05-17T04:33:15.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d0102e20 0x7f61d01052c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d01080a0 0x7f61d0105d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0112b80 con 0x7f61d0107750 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0112a00 con 0x7f61d01080a0 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d6588700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61d0112d00 con 0x7f61d0102e20 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40380/0 (socket says 172.21.15.50:40380) 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 -- 172.21.15.50:0/307703954 learned_addr learned my addr 172.21.15.50:0/307703954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cffff700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d0102e20 0x7f61d01052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61d4b25700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d01080a0 0x7f61d0105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 -- 172.21.15.50:0/307703954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d0102e20 msgr2=0x7f61d01052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d0102e20 0x7f61d01052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 -- 172.21.15.50:0/307703954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d01080a0 msgr2=0x7f61d0105d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d01080a0 0x7f61d0105d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.910+0000 7f61cf7fe700 1 -- 172.21.15.50:0/307703954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d01a8af0 con 0x7f61d0107750 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61cf7fe700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61c4000c00 tx=0x7f61c4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4027070 con 0x7f61d0107750 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c400fbc0 con 0x7f61d0107750 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4022b00 con 0x7f61d0107750 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d01a8d80 con 0x7f61d0107750 2024-05-17T04:33:15.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.911+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d01a91b0 con 0x7f61d0107750 2024-05-17T04:33:15.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.912+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d0103230 con 0x7f61d0107750 2024-05-17T04:33:15.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.914+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f61c402a070 con 0x7f61d0107750 2024-05-17T04:33:15.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.915+0000 7f61cd7fa700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 0x7f61b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:15.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.915+0000 7f61cffff700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 0x7f61b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:15.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.915+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f61c40e3660 con 0x7f61d0107750 2024-05-17T04:33:15.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.915+0000 7f61cffff700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 0x7f61b80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005950 tx=0x7f61c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:15.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:15.919+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61c4013070 con 0x7f61d0107750 2024-05-17T04:33:16.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.139+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61d0064440 con 0x7f61d0107750 2024-05-17T04:33:16.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.141+0000 7f61cd7fa700 1 -- 172.21.15.50:0/307703954 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24630 (secure 0 0 0) 0x7f61c40aaf50 con 0x7f61d0107750 2024-05-17T04:33:16.141 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:16.141 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":87,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:32:20.048458+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:33:16.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 msgr2=0x7f61b80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:16.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 0x7f61b80b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c0005950 tx=0x7f61c000b410 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 msgr2=0x7f61d0105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f61c4000c00 tx=0x7f61c4005ef0 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 shutdown_connections 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61d01080a0 0x7f61d0105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61d0102e20 0x7f61d01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f61b80b2910 0x7f61b80b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 --2- 172.21.15.50:0/307703954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61d0107750 0x7f61d0105800 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 >> 172.21.15.50:0/307703954 conn(0x7f61d0079bd0 msgr2=0x7f61d006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 shutdown_connections 2024-05-17T04:33:16.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:16.143+0000 7f61d6588700 1 -- 172.21.15.50:0/307703954 wait complete. 2024-05-17T04:33:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:16 smithi050 ceph-mon[31739]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-17T04:33:16.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:16 smithi105 ceph-mon[34795]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-17T04:33:16.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:16 smithi110 ceph-mon[35497]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-17T04:33:16.966 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T04:33:16.966 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T04:33:17.228 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:17.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:17 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/307703954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:17.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:17 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/307703954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:17.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:17 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/307703954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:18.397 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:18 smithi050 ceph-mon[31739]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T04:33:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:18 smithi105 ceph-mon[34795]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T04:33:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:18 smithi110 ceph-mon[35497]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-05-17T04:33:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 -- 172.21.15.50:0/129295478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f470810e200 msgr2=0x7f4708110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 --2- 172.21.15.50:0/129295478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f470810e200 0x7f4708110710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f470000b0a0 tx=0x7f470000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 -- 172.21.15.50:0/129295478 shutdown_connections 2024-05-17T04:33:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 --2- 172.21.15.50:0/129295478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f470810e200 0x7f4708110710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 --2- 172.21.15.50:0/129295478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4708104da0 0x7f47081051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 --2- 172.21.15.50:0/129295478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47081056f0 0x7f470810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 -- 172.21.15.50:0/129295478 >> 172.21.15.50:0/129295478 conn(0x7f4708079bd0 msgr2=0x7f4708079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 -- 172.21.15.50:0/129295478 shutdown_connections 2024-05-17T04:33:19.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.007+0000 7f470c915700 1 -- 172.21.15.50:0/129295478 wait complete. 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 Processor -- start 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 -- start start 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47081056f0 0x7f470819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f470810e200 0x7f470819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113c80 con 0x7f4708104da0 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113b00 con 0x7f470810e200 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470c915700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4708113e00 con 0x7f47081056f0 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50018/0 (socket says 172.21.15.50:50018) 2024-05-17T04:33:19.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f470659c700 1 -- 172.21.15.50:0/1051583323 learned_addr learned my addr 172.21.15.50:0/1051583323 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f4706d9d700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f470810e200 0x7f470819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.008+0000 7f4705d9b700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47081056f0 0x7f470819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 -- 172.21.15.50:0/1051583323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47081056f0 msgr2=0x7f470819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47081056f0 0x7f470819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 -- 172.21.15.50:0/1051583323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f470810e200 msgr2=0x7f470819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f470810e200 0x7f470819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 -- 172.21.15.50:0/1051583323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470819f590 con 0x7f4708104da0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470659c700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d950 tx=0x7f46f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8016070 con 0x7f4708104da0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470819f820 con 0x7f4708104da0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46f800fd20 con 0x7f4708104da0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f800e3b0 con 0x7f4708104da0 2024-05-17T04:33:19.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.009+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081a71a0 con 0x7f4708104da0 2024-05-17T04:33:19.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.010+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47081051b0 con 0x7f4708104da0 2024-05-17T04:33:19.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.012+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f46f8019070 con 0x7f4708104da0 2024-05-17T04:33:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.012+0000 7f46f77fe700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 0x7f46f00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.013+0000 7f4705d9b700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 0x7f46f00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.013+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f46f80e21a0 con 0x7f4708104da0 2024-05-17T04:33:19.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.013+0000 7f4705d9b700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 0x7f46f00b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f470810fc30 tx=0x7f46fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:19.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.017+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46f80a9a00 con 0x7f4708104da0 2024-05-17T04:33:19.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.226+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f470802ea80 con 0x7f46f00b2910 2024-05-17T04:33:19.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.233+0000 7f46f77fe700 1 -- 172.21.15.50:0/1051583323 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340622 (secure 0 0 0) 0x7f470802ea80 con 0x7f46f00b2910 2024-05-17T04:33:19.239 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 msgr2=0x7f46f00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 0x7f46f00b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f470810fc30 tx=0x7f46fc009260 comp rx=0 tx=0).stop 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 msgr2=0x7f470819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f46f800d950 tx=0x7f46f800dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 shutdown_connections 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f470810e200 0x7f470819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46f00b2910 0x7f46f00b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4708104da0 0x7f470819a6c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 --2- 172.21.15.50:0/1051583323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47081056f0 0x7f470819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:19.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 >> 172.21.15.50:0/1051583323 conn(0x7f4708079bd0 msgr2=0x7f47080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:19.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 shutdown_connections 2024-05-17T04:33:19.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:19.237+0000 7f470c915700 1 -- 172.21.15.50:0/1051583323 wait complete. 2024-05-17T04:33:19.243 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-17T04:33:20.060 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-05-17T04:33:18.729868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652801,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":254,"num_read_kb":220,"num_write":291,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373292,"kb_used_data":11660,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568532,"statfs":{"total":1151940427776,"available":1151558176768,"internally_reserved":0,"allocated":11939840,"data_stored":7889117,"data_compressed":425040,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27579,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":58,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055501+0000","last_change":"2024-05-17T04:31:42.946840+0000","last_active":"2024-05-17T04:32:20.055501+0000","last_peered":"2024-05-17T04:32:20.055501+0000","last_clean":"2024-05-17T04:32:20.055501+0000","last_became_active":"2024-05-17T04:31:42.946391+0000","last_became_peered":"2024-05-17T04:31:42.946391+0000","last_unstale":"2024-05-17T04:32:20.055501+0000","last_undegraded":"2024-05-17T04:32:20.055501+0000","last_fullsized":"2024-05-17T04:32:20.055501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:06:29.922596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052190+0000","last_change":"2024-05-17T04:31:42.934805+0000","last_active":"2024-05-17T04:32:20.052190+0000","last_peered":"2024-05-17T04:32:20.052190+0000","last_clean":"2024-05-17T04:32:20.052190+0000","last_became_active":"2024-05-17T04:31:42.933527+0000","last_became_peered":"2024-05-17T04:31:42.933527+0000","last_unstale":"2024-05-17T04:32:20.052190+0000","last_undegraded":"2024-05-17T04:32:20.052190+0000","last_fullsized":"2024-05-17T04:32:20.052190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:31.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052005+0000","last_change":"2024-05-17T04:31:42.935619+0000","last_active":"2024-05-17T04:32:20.052005+0000","last_peered":"2024-05-17T04:32:20.052005+0000","last_clean":"2024-05-17T04:32:20.052005+0000","last_became_active":"2024-05-17T04:31:42.935270+0000","last_became_peered":"2024-05-17T04:31:42.935270+0000","last_unstale":"2024-05-17T04:32:20.052005+0000","last_undegraded":"2024-05-17T04:32:20.052005+0000","last_fullsized":"2024-05-17T04:32:20.052005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:17.059859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060975+0000","last_change":"2024-05-17T04:31:42.928754+0000","last_active":"2024-05-17T04:32:20.060975+0000","last_peered":"2024-05-17T04:32:20.060975+0000","last_clean":"2024-05-17T04:32:20.060975+0000","last_became_active":"2024-05-17T04:31:42.927407+0000","last_became_peered":"2024-05-17T04:31:42.927407+0000","last_unstale":"2024-05-17T04:32:20.060975+0000","last_undegraded":"2024-05-17T04:32:20.060975+0000","last_fullsized":"2024-05-17T04:32:20.060975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:38:33.270224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052458+0000","last_change":"2024-05-17T04:31:42.929405+0000","last_active":"2024-05-17T04:32:20.052458+0000","last_peered":"2024-05-17T04:32:20.052458+0000","last_clean":"2024-05-17T04:32:20.052458+0000","last_became_active":"2024-05-17T04:31:42.928970+0000","last_became_peered":"2024-05-17T04:31:42.928970+0000","last_unstale":"2024-05-17T04:32:20.052458+0000","last_undegraded":"2024-05-17T04:32:20.052458+0000","last_fullsized":"2024-05-17T04:32:20.052458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:22:39.866681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056897+0000","last_change":"2024-05-17T04:31:42.929773+0000","last_active":"2024-05-17T04:32:20.056897+0000","last_peered":"2024-05-17T04:32:20.056897+0000","last_clean":"2024-05-17T04:32:20.056897+0000","last_became_active":"2024-05-17T04:31:42.929449+0000","last_became_peered":"2024-05-17T04:31:42.929449+0000","last_unstale":"2024-05-17T04:32:20.056897+0000","last_undegraded":"2024-05-17T04:32:20.056897+0000","last_fullsized":"2024-05-17T04:32:20.056897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:05:54.544961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059532+0000","last_change":"2024-05-17T04:31:42.930674+0000","last_active":"2024-05-17T04:32:20.059532+0000","last_peered":"2024-05-17T04:32:20.059532+0000","last_clean":"2024-05-17T04:32:20.059532+0000","last_became_active":"2024-05-17T04:31:42.929752+0000","last_became_peered":"2024-05-17T04:31:42.929752+0000","last_unstale":"2024-05-17T04:32:20.059532+0000","last_undegraded":"2024-05-17T04:32:20.059532+0000","last_fullsized":"2024-05-17T04:32:20.059532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:54:56.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197989+0000","last_change":"2024-05-17T04:31:42.941443+0000","last_active":"2024-05-17T04:32:20.197989+0000","last_peered":"2024-05-17T04:32:20.197989+0000","last_clean":"2024-05-17T04:32:20.197989+0000","last_became_active":"2024-05-17T04:31:42.941052+0000","last_became_peered":"2024-05-17T04:31:42.941052+0000","last_unstale":"2024-05-17T04:32:20.197989+0000","last_undegraded":"2024-05-17T04:32:20.197989+0000","last_fullsized":"2024-05-17T04:32:20.197989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:07.151040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054910+0000","last_change":"2024-05-17T04:31:42.938708+0000","last_active":"2024-05-17T04:32:20.054910+0000","last_peered":"2024-05-17T04:32:20.054910+0000","last_clean":"2024-05-17T04:32:20.054910+0000","last_became_active":"2024-05-17T04:31:42.938330+0000","last_became_peered":"2024-05-17T04:31:42.938330+0000","last_unstale":"2024-05-17T04:32:20.054910+0000","last_undegraded":"2024-05-17T04:32:20.054910+0000","last_fullsized":"2024-05-17T04:32:20.054910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:23.882565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056968+0000","last_change":"2024-05-17T04:31:42.932779+0000","last_active":"2024-05-17T04:32:20.056968+0000","last_peered":"2024-05-17T04:32:20.056968+0000","last_clean":"2024-05-17T04:32:20.056968+0000","last_became_active":"2024-05-17T04:31:42.932429+0000","last_became_peered":"2024-05-17T04:31:42.932429+0000","last_unstale":"2024-05-17T04:32:20.056968+0000","last_undegraded":"2024-05-17T04:32:20.056968+0000","last_fullsized":"2024-05-17T04:32:20.056968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:22:17.139775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060313+0000","last_change":"2024-05-17T04:31:42.932713+0000","last_active":"2024-05-17T04:32:20.060313+0000","last_peered":"2024-05-17T04:32:20.060313+0000","last_clean":"2024-05-17T04:32:20.060313+0000","last_became_active":"2024-05-17T04:31:42.931451+0000","last_became_peered":"2024-05-17T04:31:42.931451+0000","last_unstale":"2024-05-17T04:32:20.060313+0000","last_undegraded":"2024-05-17T04:32:20.060313+0000","last_fullsized":"2024-05-17T04:32:20.060313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:49:03.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057901+0000","last_change":"2024-05-17T04:32:20.057506+0000","last_active":"2024-05-17T04:32:20.057901+0000","last_peered":"2024-05-17T04:32:20.057901+0000","last_clean":"2024-05-17T04:32:20.057901+0000","last_became_active":"2024-05-17T04:32:20.057245+0000","last_became_peered":"2024-05-17T04:32:20.057245+0000","last_unstale":"2024-05-17T04:32:20.057901+0000","last_undegraded":"2024-05-17T04:32:20.057901+0000","last_fullsized":"2024-05-17T04:32:20.057901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:14.714699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056367+0000","last_change":"2024-05-17T04:31:42.930412+0000","last_active":"2024-05-17T04:32:20.056367+0000","last_peered":"2024-05-17T04:32:20.056367+0000","last_clean":"2024-05-17T04:32:20.056367+0000","last_became_active":"2024-05-17T04:31:42.929930+0000","last_became_peered":"2024-05-17T04:31:42.929930+0000","last_unstale":"2024-05-17T04:32:20.056367+0000","last_undegraded":"2024-05-17T04:32:20.056367+0000","last_fullsized":"2024-05-17T04:32:20.056367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:19:12.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.051688+0000","last_change":"2024-05-17T04:31:42.932696+0000","last_active":"2024-05-17T04:32:20.051688+0000","last_peered":"2024-05-17T04:32:20.051688+0000","last_clean":"2024-05-17T04:32:20.051688+0000","last_became_active":"2024-05-17T04:31:42.932391+0000","last_became_peered":"2024-05-17T04:31:42.932391+0000","last_unstale":"2024-05-17T04:32:20.051688+0000","last_undegraded":"2024-05-17T04:32:20.051688+0000","last_fullsized":"2024-05-17T04:32:20.051688+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:10:13.071352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.051607+0000","last_change":"2024-05-17T04:31:42.943331+0000","last_active":"2024-05-17T04:32:20.051607+0000","last_peered":"2024-05-17T04:32:20.051607+0000","last_clean":"2024-05-17T04:32:20.051607+0000","last_became_active":"2024-05-17T04:31:42.942963+0000","last_became_peered":"2024-05-17T04:31:42.942963+0000","last_unstale":"2024-05-17T04:32:20.051607+0000","last_undegraded":"2024-05-17T04:32:20.051607+0000","last_fullsized":"2024-05-17T04:32:20.051607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:27:33.614843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.053942+0000","last_change":"2024-05-17T04:31:42.935951+0000","last_active":"2024-05-17T04:32:20.053942+0000","last_peered":"2024-05-17T04:32:20.053942+0000","last_clean":"2024-05-17T04:32:20.053942+0000","last_became_active":"2024-05-17T04:31:42.935564+0000","last_became_peered":"2024-05-17T04:31:42.935564+0000","last_unstale":"2024-05-17T04:32:20.053942+0000","last_undegraded":"2024-05-17T04:32:20.053942+0000","last_fullsized":"2024-05-17T04:32:20.053942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:15:28.282780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055352+0000","last_change":"2024-05-17T04:31:42.946858+0000","last_active":"2024-05-17T04:32:20.055352+0000","last_peered":"2024-05-17T04:32:20.055352+0000","last_clean":"2024-05-17T04:32:20.055352+0000","last_became_active":"2024-05-17T04:31:42.946398+0000","last_became_peered":"2024-05-17T04:31:42.946398+0000","last_unstale":"2024-05-17T04:32:20.055352+0000","last_undegraded":"2024-05-17T04:32:20.055352+0000","last_fullsized":"2024-05-17T04:32:20.055352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:51:09.584967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055592+0000","last_change":"2024-05-17T04:31:42.934716+0000","last_active":"2024-05-17T04:32:20.055592+0000","last_peered":"2024-05-17T04:32:20.055592+0000","last_clean":"2024-05-17T04:32:20.055592+0000","last_became_active":"2024-05-17T04:31:42.934379+0000","last_became_peered":"2024-05-17T04:31:42.934379+0000","last_unstale":"2024-05-17T04:32:20.055592+0000","last_undegraded":"2024-05-17T04:32:20.055592+0000","last_fullsized":"2024-05-17T04:32:20.055592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:54:01.154473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057919+0000","last_change":"2024-05-17T04:31:42.926424+0000","last_active":"2024-05-17T04:32:20.057919+0000","last_peered":"2024-05-17T04:32:20.057919+0000","last_clean":"2024-05-17T04:32:20.057919+0000","last_became_active":"2024-05-17T04:31:42.925948+0000","last_became_peered":"2024-05-17T04:31:42.925948+0000","last_unstale":"2024-05-17T04:32:20.057919+0000","last_undegraded":"2024-05-17T04:32:20.057919+0000","last_fullsized":"2024-05-17T04:32:20.057919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:27:48.495395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057109+0000","last_change":"2024-05-17T04:31:55.992551+0000","last_active":"2024-05-17T04:32:20.057109+0000","last_peered":"2024-05-17T04:32:20.057109+0000","last_clean":"2024-05-17T04:32:20.057109+0000","last_became_active":"2024-05-17T04:31:55.992138+0000","last_became_peered":"2024-05-17T04:31:55.992138+0000","last_unstale":"2024-05-17T04:32:20.057109+0000","last_undegraded":"2024-05-17T04:32:20.057109+0000","last_fullsized":"2024-05-17T04:32:20.057109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:43:40.296492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.043461+0000","last_change":"2024-05-17T04:31:39.890303+0000","last_active":"2024-05-17T04:32:48.043461+0000","last_peered":"2024-05-17T04:32:48.043461+0000","last_clean":"2024-05-17T04:32:48.043461+0000","last_became_active":"2024-05-17T04:31:39.889970+0000","last_became_peered":"2024-05-17T04:31:39.889970+0000","last_unstale":"2024-05-17T04:32:48.043461+0000","last_undegraded":"2024-05-17T04:32:48.043461+0000","last_fullsized":"2024-05-17T04:32:48.043461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:29:33.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198394+0000","last_change":"2024-05-17T04:31:42.942203+0000","last_active":"2024-05-17T04:32:20.198394+0000","last_peered":"2024-05-17T04:32:20.198394+0000","last_clean":"2024-05-17T04:32:20.198394+0000","last_became_active":"2024-05-17T04:31:42.940563+0000","last_became_peered":"2024-05-17T04:31:42.940563+0000","last_unstale":"2024-05-17T04:32:20.198394+0000","last_undegraded":"2024-05-17T04:32:20.198394+0000","last_fullsized":"2024-05-17T04:32:20.198394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:19:10.813809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059148+0000","last_change":"2024-05-17T04:31:55.988355+0000","last_active":"2024-05-17T04:32:20.059148+0000","last_peered":"2024-05-17T04:32:20.059148+0000","last_clean":"2024-05-17T04:32:20.059148+0000","last_became_active":"2024-05-17T04:31:55.987874+0000","last_became_peered":"2024-05-17T04:31:55.987874+0000","last_unstale":"2024-05-17T04:32:20.059148+0000","last_undegraded":"2024-05-17T04:32:20.059148+0000","last_fullsized":"2024-05-17T04:32:20.059148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:13:30.650959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198554+0000","last_change":"2024-05-17T04:31:39.896754+0000","last_active":"2024-05-17T04:32:20.198554+0000","last_peered":"2024-05-17T04:32:20.198554+0000","last_clean":"2024-05-17T04:32:20.198554+0000","last_became_active":"2024-05-17T04:31:39.896369+0000","last_became_peered":"2024-05-17T04:31:39.896369+0000","last_unstale":"2024-05-17T04:32:20.198554+0000","last_undegraded":"2024-05-17T04:32:20.198554+0000","last_fullsized":"2024-05-17T04:32:20.198554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:33:03.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052339+0000","last_change":"2024-05-17T04:31:42.938556+0000","last_active":"2024-05-17T04:32:20.052339+0000","last_peered":"2024-05-17T04:32:20.052339+0000","last_clean":"2024-05-17T04:32:20.052339+0000","last_became_active":"2024-05-17T04:31:42.938228+0000","last_became_peered":"2024-05-17T04:31:42.938228+0000","last_unstale":"2024-05-17T04:32:20.052339+0000","last_undegraded":"2024-05-17T04:32:20.052339+0000","last_fullsized":"2024-05-17T04:32:20.052339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:18:32.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059222+0000","last_change":"2024-05-17T04:31:55.991620+0000","last_active":"2024-05-17T04:32:20.059222+0000","last_peered":"2024-05-17T04:32:20.059222+0000","last_clean":"2024-05-17T04:32:20.059222+0000","last_became_active":"2024-05-17T04:31:55.990762+0000","last_became_peered":"2024-05-17T04:31:55.990762+0000","last_unstale":"2024-05-17T04:32:20.059222+0000","last_undegraded":"2024-05-17T04:32:20.059222+0000","last_fullsized":"2024-05-17T04:32:20.059222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:22:49.033835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.041261+0000","last_change":"2024-05-17T04:32:20.067654+0000","last_active":"2024-05-17T04:32:48.041261+0000","last_peered":"2024-05-17T04:32:48.041261+0000","last_clean":"2024-05-17T04:32:48.041261+0000","last_became_active":"2024-05-17T04:32:20.062315+0000","last_became_peered":"2024-05-17T04:32:20.062315+0000","last_unstale":"2024-05-17T04:32:48.041261+0000","last_undegraded":"2024-05-17T04:32:48.041261+0000","last_fullsized":"2024-05-17T04:32:48.041261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:32:52.479838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054655+0000","last_change":"2024-05-17T04:31:42.938076+0000","last_active":"2024-05-17T04:32:20.054655+0000","last_peered":"2024-05-17T04:32:20.054655+0000","last_clean":"2024-05-17T04:32:20.054655+0000","last_became_active":"2024-05-17T04:31:42.937177+0000","last_became_peered":"2024-05-17T04:31:42.937177+0000","last_unstale":"2024-05-17T04:32:20.054655+0000","last_undegraded":"2024-05-17T04:32:20.054655+0000","last_fullsized":"2024-05-17T04:32:20.054655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:08:29.759076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058302+0000","last_change":"2024-05-17T04:31:55.988774+0000","last_active":"2024-05-17T04:32:20.058302+0000","last_peered":"2024-05-17T04:32:20.058302+0000","last_clean":"2024-05-17T04:32:20.058302+0000","last_became_active":"2024-05-17T04:31:55.987934+0000","last_became_peered":"2024-05-17T04:31:55.987934+0000","last_unstale":"2024-05-17T04:32:20.058302+0000","last_undegraded":"2024-05-17T04:32:20.058302+0000","last_fullsized":"2024-05-17T04:32:20.058302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:24:53.408890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.053099+0000","last_change":"2024-05-17T04:31:39.896382+0000","last_active":"2024-05-17T04:32:20.053099+0000","last_peered":"2024-05-17T04:32:20.053099+0000","last_clean":"2024-05-17T04:32:20.053099+0000","last_became_active":"2024-05-17T04:31:39.895863+0000","last_became_peered":"2024-05-17T04:31:39.895863+0000","last_unstale":"2024-05-17T04:32:20.053099+0000","last_undegraded":"2024-05-17T04:32:20.053099+0000","last_fullsized":"2024-05-17T04:32:20.053099+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:08:37.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060759+0000","last_change":"2024-05-17T04:31:42.938624+0000","last_active":"2024-05-17T04:32:20.060759+0000","last_peered":"2024-05-17T04:32:20.060759+0000","last_clean":"2024-05-17T04:32:20.060759+0000","last_became_active":"2024-05-17T04:31:42.938197+0000","last_became_peered":"2024-05-17T04:31:42.938197+0000","last_unstale":"2024-05-17T04:32:20.060759+0000","last_undegraded":"2024-05-17T04:32:20.060759+0000","last_fullsized":"2024-05-17T04:32:20.060759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:47:14.474259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057672+0000","last_change":"2024-05-17T04:31:55.988254+0000","last_active":"2024-05-17T04:32:20.057672+0000","last_peered":"2024-05-17T04:32:20.057672+0000","last_clean":"2024-05-17T04:32:20.057672+0000","last_became_active":"2024-05-17T04:31:55.987790+0000","last_became_peered":"2024-05-17T04:31:55.987790+0000","last_unstale":"2024-05-17T04:32:20.057672+0000","last_undegraded":"2024-05-17T04:32:20.057672+0000","last_fullsized":"2024-05-17T04:32:20.057672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:07:03.189403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:12.090684+0000","last_change":"2024-05-17T04:31:39.894789+0000","last_active":"2024-05-17T04:33:12.090684+0000","last_peered":"2024-05-17T04:33:12.090684+0000","last_clean":"2024-05-17T04:33:12.090684+0000","last_became_active":"2024-05-17T04:31:39.894404+0000","last_became_peered":"2024-05-17T04:31:39.894404+0000","last_unstale":"2024-05-17T04:33:12.090684+0000","last_undegraded":"2024-05-17T04:33:12.090684+0000","last_fullsized":"2024-05-17T04:33:12.090684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:00:25.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052446+0000","last_change":"2024-05-17T04:31:42.932430+0000","last_active":"2024-05-17T04:32:20.052446+0000","last_peered":"2024-05-17T04:32:20.052446+0000","last_clean":"2024-05-17T04:32:20.052446+0000","last_became_active":"2024-05-17T04:31:42.932039+0000","last_became_peered":"2024-05-17T04:31:42.932039+0000","last_unstale":"2024-05-17T04:32:20.052446+0000","last_undegraded":"2024-05-17T04:32:20.052446+0000","last_fullsized":"2024-05-17T04:32:20.052446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:46:53.928092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057284+0000","last_change":"2024-05-17T04:31:55.992898+0000","last_active":"2024-05-17T04:32:20.057284+0000","last_peered":"2024-05-17T04:32:20.057284+0000","last_clean":"2024-05-17T04:32:20.057284+0000","last_became_active":"2024-05-17T04:31:55.992459+0000","last_became_peered":"2024-05-17T04:31:55.992459+0000","last_unstale":"2024-05-17T04:32:20.057284+0000","last_undegraded":"2024-05-17T04:32:20.057284+0000","last_fullsized":"2024-05-17T04:32:20.057284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:45:24.888533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.040360+0000","last_change":"2024-05-17T04:31:39.893014+0000","last_active":"2024-05-17T04:32:48.040360+0000","last_peered":"2024-05-17T04:32:48.040360+0000","last_clean":"2024-05-17T04:32:48.040360+0000","last_became_active":"2024-05-17T04:31:39.892602+0000","last_became_peered":"2024-05-17T04:31:39.892602+0000","last_unstale":"2024-05-17T04:32:48.040360+0000","last_undegraded":"2024-05-17T04:32:48.040360+0000","last_fullsized":"2024-05-17T04:32:48.040360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:45:12.742516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061404+0000","last_change":"2024-05-17T04:31:42.945750+0000","last_active":"2024-05-17T04:32:20.061404+0000","last_peered":"2024-05-17T04:32:20.061404+0000","last_clean":"2024-05-17T04:32:20.061404+0000","last_became_active":"2024-05-17T04:31:42.945332+0000","last_became_peered":"2024-05-17T04:31:42.945332+0000","last_unstale":"2024-05-17T04:32:20.061404+0000","last_undegraded":"2024-05-17T04:32:20.061404+0000","last_fullsized":"2024-05-17T04:32:20.061404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:36:00.521489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055211+0000","last_change":"2024-05-17T04:31:55.984513+0000","last_active":"2024-05-17T04:32:20.055211+0000","last_peered":"2024-05-17T04:32:20.055211+0000","last_clean":"2024-05-17T04:32:20.055211+0000","last_became_active":"2024-05-17T04:31:55.984065+0000","last_became_peered":"2024-05-17T04:31:55.984065+0000","last_unstale":"2024-05-17T04:32:20.055211+0000","last_undegraded":"2024-05-17T04:32:20.055211+0000","last_fullsized":"2024-05-17T04:32:20.055211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:07:06.370969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.042200+0000","last_change":"2024-05-17T04:31:39.889551+0000","last_active":"2024-05-17T04:32:48.042200+0000","last_peered":"2024-05-17T04:32:48.042200+0000","last_clean":"2024-05-17T04:32:48.042200+0000","last_became_active":"2024-05-17T04:31:39.889150+0000","last_became_peered":"2024-05-17T04:31:39.889150+0000","last_unstale":"2024-05-17T04:32:48.042200+0000","last_undegraded":"2024-05-17T04:32:48.042200+0000","last_fullsized":"2024-05-17T04:32:48.042200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:14:06.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198119+0000","last_change":"2024-05-17T04:31:42.922236+0000","last_active":"2024-05-17T04:32:20.198119+0000","last_peered":"2024-05-17T04:32:20.198119+0000","last_clean":"2024-05-17T04:32:20.198119+0000","last_became_active":"2024-05-17T04:31:42.921907+0000","last_became_peered":"2024-05-17T04:31:42.921907+0000","last_unstale":"2024-05-17T04:32:20.198119+0000","last_undegraded":"2024-05-17T04:32:20.198119+0000","last_fullsized":"2024-05-17T04:32:20.198119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:10:09.941419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052170+0000","last_change":"2024-05-17T04:31:55.997210+0000","last_active":"2024-05-17T04:32:20.052170+0000","last_peered":"2024-05-17T04:32:20.052170+0000","last_clean":"2024-05-17T04:32:20.052170+0000","last_became_active":"2024-05-17T04:31:55.996795+0000","last_became_peered":"2024-05-17T04:31:55.996795+0000","last_unstale":"2024-05-17T04:32:20.052170+0000","last_undegraded":"2024-05-17T04:32:20.052170+0000","last_fullsized":"2024-05-17T04:32:20.052170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:51:42.271149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060686+0000","last_change":"2024-05-17T04:31:39.891980+0000","last_active":"2024-05-17T04:32:20.060686+0000","last_peered":"2024-05-17T04:32:20.060686+0000","last_clean":"2024-05-17T04:32:20.060686+0000","last_became_active":"2024-05-17T04:31:39.891636+0000","last_became_peered":"2024-05-17T04:31:39.891636+0000","last_unstale":"2024-05-17T04:32:20.060686+0000","last_undegraded":"2024-05-17T04:32:20.060686+0000","last_fullsized":"2024-05-17T04:32:20.060686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:05:33.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055092+0000","last_change":"2024-05-17T04:31:42.929970+0000","last_active":"2024-05-17T04:32:20.055092+0000","last_peered":"2024-05-17T04:32:20.055092+0000","last_clean":"2024-05-17T04:32:20.055092+0000","last_became_active":"2024-05-17T04:31:42.929587+0000","last_became_peered":"2024-05-17T04:31:42.929587+0000","last_unstale":"2024-05-17T04:32:20.055092+0000","last_undegraded":"2024-05-17T04:32:20.055092+0000","last_fullsized":"2024-05-17T04:32:20.055092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:05:38.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197931+0000","last_change":"2024-05-17T04:31:55.991388+0000","last_active":"2024-05-17T04:32:20.197931+0000","last_peered":"2024-05-17T04:32:20.197931+0000","last_clean":"2024-05-17T04:32:20.197931+0000","last_became_active":"2024-05-17T04:31:55.990979+0000","last_became_peered":"2024-05-17T04:31:55.990979+0000","last_unstale":"2024-05-17T04:32:20.197931+0000","last_undegraded":"2024-05-17T04:32:20.197931+0000","last_fullsized":"2024-05-17T04:32:20.197931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:34.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:13.091545+0000","last_change":"2024-05-17T04:31:39.893076+0000","last_active":"2024-05-17T04:33:13.091545+0000","last_peered":"2024-05-17T04:33:13.091545+0000","last_clean":"2024-05-17T04:33:13.091545+0000","last_became_active":"2024-05-17T04:31:39.892683+0000","last_became_peered":"2024-05-17T04:31:39.892683+0000","last_unstale":"2024-05-17T04:33:13.091545+0000","last_undegraded":"2024-05-17T04:33:13.091545+0000","last_fullsized":"2024-05-17T04:33:13.091545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:32:19.583662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054806+0000","last_change":"2024-05-17T04:31:42.938431+0000","last_active":"2024-05-17T04:32:20.054806+0000","last_peered":"2024-05-17T04:32:20.054806+0000","last_clean":"2024-05-17T04:32:20.054806+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:32:20.054806+0000","last_undegraded":"2024-05-17T04:32:20.054806+0000","last_fullsized":"2024-05-17T04:32:20.054806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:56:34.505923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.066417+0000","last_change":"2024-05-17T04:32:20.065723+0000","last_active":"2024-05-17T04:32:20.066417+0000","last_peered":"2024-05-17T04:32:20.066417+0000","last_clean":"2024-05-17T04:32:20.066417+0000","last_became_active":"2024-05-17T04:32:20.065266+0000","last_became_peered":"2024-05-17T04:32:20.065266+0000","last_unstale":"2024-05-17T04:32:20.066417+0000","last_undegraded":"2024-05-17T04:32:20.066417+0000","last_fullsized":"2024-05-17T04:32:20.066417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:16:54.256206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036246+0000","last_change":"2024-05-17T04:31:39.892324+0000","last_active":"2024-05-17T04:32:47.036246+0000","last_peered":"2024-05-17T04:32:47.036246+0000","last_clean":"2024-05-17T04:32:47.036246+0000","last_became_active":"2024-05-17T04:31:39.891398+0000","last_became_peered":"2024-05-17T04:31:39.891398+0000","last_unstale":"2024-05-17T04:32:47.036246+0000","last_undegraded":"2024-05-17T04:32:47.036246+0000","last_fullsized":"2024-05-17T04:32:47.036246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:06.423894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052217+0000","last_change":"2024-05-17T04:31:42.948320+0000","last_active":"2024-05-17T04:32:20.052217+0000","last_peered":"2024-05-17T04:32:20.052217+0000","last_clean":"2024-05-17T04:32:20.052217+0000","last_became_active":"2024-05-17T04:31:42.948009+0000","last_became_peered":"2024-05-17T04:31:42.948009+0000","last_unstale":"2024-05-17T04:32:20.052217+0000","last_undegraded":"2024-05-17T04:32:20.052217+0000","last_fullsized":"2024-05-17T04:32:20.052217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:51:28.464925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059573+0000","last_change":"2024-05-17T04:31:55.984602+0000","last_active":"2024-05-17T04:32:20.059573+0000","last_peered":"2024-05-17T04:32:20.059573+0000","last_clean":"2024-05-17T04:32:20.059573+0000","last_became_active":"2024-05-17T04:31:55.984152+0000","last_became_peered":"2024-05-17T04:31:55.984152+0000","last_unstale":"2024-05-17T04:32:20.059573+0000","last_undegraded":"2024-05-17T04:32:20.059573+0000","last_fullsized":"2024-05-17T04:32:20.059573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:31:23.456878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036130+0000","last_change":"2024-05-17T04:31:39.896889+0000","last_active":"2024-05-17T04:32:47.036130+0000","last_peered":"2024-05-17T04:32:47.036130+0000","last_clean":"2024-05-17T04:32:47.036130+0000","last_became_active":"2024-05-17T04:31:39.896516+0000","last_became_peered":"2024-05-17T04:31:39.896516+0000","last_unstale":"2024-05-17T04:32:47.036130+0000","last_undegraded":"2024-05-17T04:32:47.036130+0000","last_fullsized":"2024-05-17T04:32:47.036130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:45:46.214589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054888+0000","last_change":"2024-05-17T04:31:42.928588+0000","last_active":"2024-05-17T04:32:20.054888+0000","last_peered":"2024-05-17T04:32:20.054888+0000","last_clean":"2024-05-17T04:32:20.054888+0000","last_became_active":"2024-05-17T04:31:42.928215+0000","last_became_peered":"2024-05-17T04:31:42.928215+0000","last_unstale":"2024-05-17T04:32:20.054888+0000","last_undegraded":"2024-05-17T04:32:20.054888+0000","last_fullsized":"2024-05-17T04:32:20.054888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:49:31.894516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.063156+0000","last_change":"2024-05-17T04:32:20.063156+0000","last_active":"2024-05-17T04:32:20.063156+0000","last_peered":"2024-05-17T04:32:20.063156+0000","last_clean":"2024-05-17T04:32:20.063156+0000","last_became_active":"2024-05-17T04:32:20.062600+0000","last_became_peered":"2024-05-17T04:32:20.062600+0000","last_unstale":"2024-05-17T04:32:20.063156+0000","last_undegraded":"2024-05-17T04:32:20.063156+0000","last_fullsized":"2024-05-17T04:32:20.063156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:58:04.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:14.093068+0000","last_change":"2024-05-17T04:31:39.896849+0000","last_active":"2024-05-17T04:33:14.093068+0000","last_peered":"2024-05-17T04:33:14.093068+0000","last_clean":"2024-05-17T04:33:14.093068+0000","last_became_active":"2024-05-17T04:31:39.896134+0000","last_became_peered":"2024-05-17T04:31:39.896134+0000","last_unstale":"2024-05-17T04:33:14.093068+0000","last_undegraded":"2024-05-17T04:33:14.093068+0000","last_fullsized":"2024-05-17T04:33:14.093068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:00:27.150148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":85,"num_write":33,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052956+0000","last_change":"2024-05-17T04:31:42.932473+0000","last_active":"2024-05-17T04:32:20.052956+0000","last_peered":"2024-05-17T04:32:20.052956+0000","last_clean":"2024-05-17T04:32:20.052956+0000","last_became_active":"2024-05-17T04:31:42.932120+0000","last_became_peered":"2024-05-17T04:31:42.932120+0000","last_unstale":"2024-05-17T04:32:20.052956+0000","last_undegraded":"2024-05-17T04:32:20.052956+0000","last_fullsized":"2024-05-17T04:32:20.052956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:57:35.056071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061085+0000","last_change":"2024-05-17T04:31:55.999678+0000","last_active":"2024-05-17T04:32:20.061085+0000","last_peered":"2024-05-17T04:32:20.061085+0000","last_clean":"2024-05-17T04:32:20.061085+0000","last_became_active":"2024-05-17T04:31:55.999236+0000","last_became_peered":"2024-05-17T04:31:55.999236+0000","last_unstale":"2024-05-17T04:32:20.061085+0000","last_undegraded":"2024-05-17T04:32:20.061085+0000","last_fullsized":"2024-05-17T04:32:20.061085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:16:27.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.040339+0000","last_change":"2024-05-17T04:31:39.894334+0000","last_active":"2024-05-17T04:32:48.040339+0000","last_peered":"2024-05-17T04:32:48.040339+0000","last_clean":"2024-05-17T04:32:48.040339+0000","last_became_active":"2024-05-17T04:31:39.893801+0000","last_became_peered":"2024-05-17T04:31:39.893801+0000","last_unstale":"2024-05-17T04:32:48.040339+0000","last_undegraded":"2024-05-17T04:32:48.040339+0000","last_fullsized":"2024-05-17T04:32:48.040339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:26:07.143110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055984+0000","last_change":"2024-05-17T04:31:42.934598+0000","last_active":"2024-05-17T04:32:20.055984+0000","last_peered":"2024-05-17T04:32:20.055984+0000","last_clean":"2024-05-17T04:32:20.055984+0000","last_became_active":"2024-05-17T04:31:42.934268+0000","last_became_peered":"2024-05-17T04:31:42.934268+0000","last_unstale":"2024-05-17T04:32:20.055984+0000","last_undegraded":"2024-05-17T04:32:20.055984+0000","last_fullsized":"2024-05-17T04:32:20.055984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:50:13.868834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054952+0000","last_change":"2024-05-17T04:31:55.986888+0000","last_active":"2024-05-17T04:32:20.054952+0000","last_peered":"2024-05-17T04:32:20.054952+0000","last_clean":"2024-05-17T04:32:20.054952+0000","last_became_active":"2024-05-17T04:31:55.986369+0000","last_became_peered":"2024-05-17T04:31:55.986369+0000","last_unstale":"2024-05-17T04:32:20.054952+0000","last_undegraded":"2024-05-17T04:32:20.054952+0000","last_fullsized":"2024-05-17T04:32:20.054952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:42:40.672724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037676+0000","last_change":"2024-05-17T04:31:39.891226+0000","last_active":"2024-05-17T04:32:47.037676+0000","last_peered":"2024-05-17T04:32:47.037676+0000","last_clean":"2024-05-17T04:32:47.037676+0000","last_became_active":"2024-05-17T04:31:39.890376+0000","last_became_peered":"2024-05-17T04:31:39.890376+0000","last_unstale":"2024-05-17T04:32:47.037676+0000","last_undegraded":"2024-05-17T04:32:47.037676+0000","last_fullsized":"2024-05-17T04:32:47.037676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:13:14.749546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.062998+0000","last_change":"2024-05-17T04:32:20.062998+0000","last_active":"2024-05-17T04:32:20.062998+0000","last_peered":"2024-05-17T04:32:20.062998+0000","last_clean":"2024-05-17T04:32:20.062998+0000","last_became_active":"2024-05-17T04:32:20.062485+0000","last_became_peered":"2024-05-17T04:32:20.062485+0000","last_unstale":"2024-05-17T04:32:20.062998+0000","last_undegraded":"2024-05-17T04:32:20.062998+0000","last_fullsized":"2024-05-17T04:32:20.062998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:20:23.822735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055417+0000","last_change":"2024-05-17T04:31:55.994500+0000","last_active":"2024-05-17T04:32:20.055417+0000","last_peered":"2024-05-17T04:32:20.055417+0000","last_clean":"2024-05-17T04:32:20.055417+0000","last_became_active":"2024-05-17T04:31:55.994130+0000","last_became_peered":"2024-05-17T04:31:55.994130+0000","last_unstale":"2024-05-17T04:32:20.055417+0000","last_undegraded":"2024-05-17T04:32:20.055417+0000","last_fullsized":"2024-05-17T04:32:20.055417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:15:33.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:13.091698+0000","last_change":"2024-05-17T04:31:39.894453+0000","last_active":"2024-05-17T04:33:13.091698+0000","last_peered":"2024-05-17T04:33:13.091698+0000","last_clean":"2024-05-17T04:33:13.091698+0000","last_became_active":"2024-05-17T04:31:39.893628+0000","last_became_peered":"2024-05-17T04:31:39.893628+0000","last_unstale":"2024-05-17T04:33:13.091698+0000","last_undegraded":"2024-05-17T04:33:13.091698+0000","last_fullsized":"2024-05-17T04:33:13.091698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:55:48.098182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059157+0000","last_change":"2024-05-17T04:31:42.939487+0000","last_active":"2024-05-17T04:32:20.059157+0000","last_peered":"2024-05-17T04:32:20.059157+0000","last_clean":"2024-05-17T04:32:20.059157+0000","last_became_active":"2024-05-17T04:31:42.938719+0000","last_became_peered":"2024-05-17T04:31:42.938719+0000","last_unstale":"2024-05-17T04:32:20.059157+0000","last_undegraded":"2024-05-17T04:32:20.059157+0000","last_fullsized":"2024-05-17T04:32:20.059157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:22:42.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056715+0000","last_change":"2024-05-17T04:31:55.998287+0000","last_active":"2024-05-17T04:32:20.056715+0000","last_peered":"2024-05-17T04:32:20.056715+0000","last_clean":"2024-05-17T04:32:20.056715+0000","last_became_active":"2024-05-17T04:31:55.997876+0000","last_became_peered":"2024-05-17T04:31:55.997876+0000","last_unstale":"2024-05-17T04:32:20.056715+0000","last_undegraded":"2024-05-17T04:32:20.056715+0000","last_fullsized":"2024-05-17T04:32:20.056715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:20.523375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.043308+0000","last_change":"2024-05-17T04:31:39.896995+0000","last_active":"2024-05-17T04:32:48.043308+0000","last_peered":"2024-05-17T04:32:48.043308+0000","last_clean":"2024-05-17T04:32:48.043308+0000","last_became_active":"2024-05-17T04:31:39.896607+0000","last_became_peered":"2024-05-17T04:31:39.896607+0000","last_unstale":"2024-05-17T04:32:48.043308+0000","last_undegraded":"2024-05-17T04:32:48.043308+0000","last_fullsized":"2024-05-17T04:32:48.043308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:13:26.761427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061923+0000","last_change":"2024-05-17T04:32:20.061507+0000","last_active":"2024-05-17T04:32:20.061923+0000","last_peered":"2024-05-17T04:32:20.061923+0000","last_clean":"2024-05-17T04:32:20.061923+0000","last_became_active":"2024-05-17T04:32:20.061245+0000","last_became_peered":"2024-05-17T04:32:20.061245+0000","last_unstale":"2024-05-17T04:32:20.061923+0000","last_undegraded":"2024-05-17T04:32:20.061923+0000","last_fullsized":"2024-05-17T04:32:20.061923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:43.800524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058971+0000","last_change":"2024-05-17T04:31:55.991340+0000","last_active":"2024-05-17T04:32:20.058971+0000","last_peered":"2024-05-17T04:32:20.058971+0000","last_clean":"2024-05-17T04:32:20.058971+0000","last_became_active":"2024-05-17T04:31:55.990733+0000","last_became_peered":"2024-05-17T04:31:55.990733+0000","last_unstale":"2024-05-17T04:32:20.058971+0000","last_undegraded":"2024-05-17T04:32:20.058971+0000","last_fullsized":"2024-05-17T04:32:20.058971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:01:47.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:50.037510+0000","last_change":"2024-05-17T04:32:20.070153+0000","last_active":"2024-05-17T04:32:50.037510+0000","last_peered":"2024-05-17T04:32:50.037510+0000","last_clean":"2024-05-17T04:32:50.037510+0000","last_became_active":"2024-05-17T04:32:20.062400+0000","last_became_peered":"2024-05-17T04:32:20.062400+0000","last_unstale":"2024-05-17T04:32:50.037510+0000","last_undegraded":"2024-05-17T04:32:50.037510+0000","last_fullsized":"2024-05-17T04:32:50.037510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:25:03.332079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.064121+0000","last_change":"2024-05-17T04:32:20.064121+0000","last_active":"2024-05-17T04:32:20.064121+0000","last_peered":"2024-05-17T04:32:20.064121+0000","last_clean":"2024-05-17T04:32:20.064121+0000","last_became_active":"2024-05-17T04:32:20.063707+0000","last_became_peered":"2024-05-17T04:32:20.063707+0000","last_unstale":"2024-05-17T04:32:20.064121+0000","last_undegraded":"2024-05-17T04:32:20.064121+0000","last_fullsized":"2024-05-17T04:32:20.064121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:57:45.362980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052409+0000","last_change":"2024-05-17T04:31:56.178977+0000","last_active":"2024-05-17T04:32:20.052409+0000","last_peered":"2024-05-17T04:32:20.052409+0000","last_clean":"2024-05-17T04:32:20.052409+0000","last_became_active":"2024-05-17T04:31:56.178452+0000","last_became_peered":"2024-05-17T04:31:56.178452+0000","last_unstale":"2024-05-17T04:32:20.052409+0000","last_undegraded":"2024-05-17T04:32:20.052409+0000","last_fullsized":"2024-05-17T04:32:20.052409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:14:19.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055838+0000","last_change":"2024-05-17T04:31:39.897118+0000","last_active":"2024-05-17T04:32:20.055838+0000","last_peered":"2024-05-17T04:32:20.055838+0000","last_clean":"2024-05-17T04:32:20.055838+0000","last_became_active":"2024-05-17T04:31:39.896761+0000","last_became_peered":"2024-05-17T04:31:39.896761+0000","last_unstale":"2024-05-17T04:32:20.055838+0000","last_undegraded":"2024-05-17T04:32:20.055838+0000","last_fullsized":"2024-05-17T04:32:20.055838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:41:50.575131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055597+0000","last_change":"2024-05-17T04:31:42.930510+0000","last_active":"2024-05-17T04:32:20.055597+0000","last_peered":"2024-05-17T04:32:20.055597+0000","last_clean":"2024-05-17T04:32:20.055597+0000","last_became_active":"2024-05-17T04:31:42.930168+0000","last_became_peered":"2024-05-17T04:31:42.930168+0000","last_unstale":"2024-05-17T04:32:20.055597+0000","last_undegraded":"2024-05-17T04:32:20.055597+0000","last_fullsized":"2024-05-17T04:32:20.055597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:18:23.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056521+0000","last_change":"2024-05-17T04:31:55.996783+0000","last_active":"2024-05-17T04:32:20.056521+0000","last_peered":"2024-05-17T04:32:20.056521+0000","last_clean":"2024-05-17T04:32:20.056521+0000","last_became_active":"2024-05-17T04:31:55.996343+0000","last_became_peered":"2024-05-17T04:31:55.996343+0000","last_unstale":"2024-05-17T04:32:20.056521+0000","last_undegraded":"2024-05-17T04:32:20.056521+0000","last_fullsized":"2024-05-17T04:32:20.056521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:41:49.074496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060706+0000","last_change":"2024-05-17T04:31:39.890854+0000","last_active":"2024-05-17T04:32:20.060706+0000","last_peered":"2024-05-17T04:32:20.060706+0000","last_clean":"2024-05-17T04:32:20.060706+0000","last_became_active":"2024-05-17T04:31:39.890379+0000","last_became_peered":"2024-05-17T04:31:39.890379+0000","last_unstale":"2024-05-17T04:32:20.060706+0000","last_undegraded":"2024-05-17T04:32:20.060706+0000","last_fullsized":"2024-05-17T04:32:20.060706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:06.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055604+0000","last_change":"2024-05-17T04:31:42.936992+0000","last_active":"2024-05-17T04:32:20.055604+0000","last_peered":"2024-05-17T04:32:20.055604+0000","last_clean":"2024-05-17T04:32:20.055604+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:32:20.055604+0000","last_undegraded":"2024-05-17T04:32:20.055604+0000","last_fullsized":"2024-05-17T04:32:20.055604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:27:33.067173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056460+0000","last_change":"2024-05-17T04:31:56.178921+0000","last_active":"2024-05-17T04:32:20.056460+0000","last_peered":"2024-05-17T04:32:20.056460+0000","last_clean":"2024-05-17T04:32:20.056460+0000","last_became_active":"2024-05-17T04:31:56.178430+0000","last_became_peered":"2024-05-17T04:31:56.178430+0000","last_unstale":"2024-05-17T04:32:20.056460+0000","last_undegraded":"2024-05-17T04:32:20.056460+0000","last_fullsized":"2024-05-17T04:32:20.056460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:56:37.980197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060976+0000","last_change":"2024-05-17T04:31:42.938586+0000","last_active":"2024-05-17T04:32:20.060976+0000","last_peered":"2024-05-17T04:32:20.060976+0000","last_clean":"2024-05-17T04:32:20.060976+0000","last_became_active":"2024-05-17T04:31:42.937861+0000","last_became_peered":"2024-05-17T04:31:42.937861+0000","last_unstale":"2024-05-17T04:32:20.060976+0000","last_undegraded":"2024-05-17T04:32:20.060976+0000","last_fullsized":"2024-05-17T04:32:20.060976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:03:11.790930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:14.092488+0000","last_change":"2024-05-17T04:31:39.896824+0000","last_active":"2024-05-17T04:33:14.092488+0000","last_peered":"2024-05-17T04:33:14.092488+0000","last_clean":"2024-05-17T04:33:14.092488+0000","last_became_active":"2024-05-17T04:31:39.896463+0000","last_became_peered":"2024-05-17T04:31:39.896463+0000","last_unstale":"2024-05-17T04:33:14.092488+0000","last_undegraded":"2024-05-17T04:33:14.092488+0000","last_fullsized":"2024-05-17T04:33:14.092488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:36:08.351784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060596+0000","last_change":"2024-05-17T04:31:55.993336+0000","last_active":"2024-05-17T04:32:20.060596+0000","last_peered":"2024-05-17T04:32:20.060596+0000","last_clean":"2024-05-17T04:32:20.060596+0000","last_became_active":"2024-05-17T04:31:55.992851+0000","last_became_peered":"2024-05-17T04:31:55.992851+0000","last_unstale":"2024-05-17T04:32:20.060596+0000","last_undegraded":"2024-05-17T04:32:20.060596+0000","last_fullsized":"2024-05-17T04:32:20.060596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:07:47.563166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198117+0000","last_change":"2024-05-17T04:31:42.941862+0000","last_active":"2024-05-17T04:32:20.198117+0000","last_peered":"2024-05-17T04:32:20.198117+0000","last_clean":"2024-05-17T04:32:20.198117+0000","last_became_active":"2024-05-17T04:31:42.940304+0000","last_became_peered":"2024-05-17T04:31:42.940304+0000","last_unstale":"2024-05-17T04:32:20.198117+0000","last_undegraded":"2024-05-17T04:32:20.198117+0000","last_fullsized":"2024-05-17T04:32:20.198117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:46:14.569208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:50.052979+0000","last_change":"2024-05-17T04:31:39.894080+0000","last_active":"2024-05-17T04:32:50.052979+0000","last_peered":"2024-05-17T04:32:50.052979+0000","last_clean":"2024-05-17T04:32:50.052979+0000","last_became_active":"2024-05-17T04:31:39.893646+0000","last_became_peered":"2024-05-17T04:31:39.893646+0000","last_unstale":"2024-05-17T04:32:50.052979+0000","last_undegraded":"2024-05-17T04:32:50.052979+0000","last_fullsized":"2024-05-17T04:32:50.052979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:27:59.844497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058908+0000","last_change":"2024-05-17T04:30:11.017843+0000","last_active":"2024-05-17T04:32:20.058908+0000","last_peered":"2024-05-17T04:32:20.058908+0000","last_clean":"2024-05-17T04:32:20.058908+0000","last_became_active":"2024-05-17T04:30:11.013443+0000","last_became_peered":"2024-05-17T04:30:11.013443+0000","last_unstale":"2024-05-17T04:32:20.058908+0000","last_undegraded":"2024-05-17T04:32:20.058908+0000","last_fullsized":"2024-05-17T04:32:20.058908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_clean_scrub_stamp":"2024-05-17T04:26:43.066168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:27:18.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061659+0000","last_change":"2024-05-17T04:31:56.179071+0000","last_active":"2024-05-17T04:32:20.061659+0000","last_peered":"2024-05-17T04:32:20.061659+0000","last_clean":"2024-05-17T04:32:20.061659+0000","last_became_active":"2024-05-17T04:31:56.178500+0000","last_became_peered":"2024-05-17T04:31:56.178500+0000","last_unstale":"2024-05-17T04:32:20.061659+0000","last_undegraded":"2024-05-17T04:32:20.061659+0000","last_fullsized":"2024-05-17T04:32:20.061659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:48:02.144119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055528+0000","last_change":"2024-05-17T04:31:42.932232+0000","last_active":"2024-05-17T04:32:20.055528+0000","last_peered":"2024-05-17T04:32:20.055528+0000","last_clean":"2024-05-17T04:32:20.055528+0000","last_became_active":"2024-05-17T04:31:42.931915+0000","last_became_peered":"2024-05-17T04:31:42.931915+0000","last_unstale":"2024-05-17T04:32:20.055528+0000","last_undegraded":"2024-05-17T04:32:20.055528+0000","last_fullsized":"2024-05-17T04:32:20.055528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:06:57.693180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.041239+0000","last_change":"2024-05-17T04:31:39.890852+0000","last_active":"2024-05-17T04:32:48.041239+0000","last_peered":"2024-05-17T04:32:48.041239+0000","last_clean":"2024-05-17T04:32:48.041239+0000","last_became_active":"2024-05-17T04:31:39.890417+0000","last_became_peered":"2024-05-17T04:31:39.890417+0000","last_unstale":"2024-05-17T04:32:48.041239+0000","last_undegraded":"2024-05-17T04:32:48.041239+0000","last_fullsized":"2024-05-17T04:32:48.041239+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:45:47.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054730+0000","last_change":"2024-05-17T04:31:55.988931+0000","last_active":"2024-05-17T04:32:20.054730+0000","last_peered":"2024-05-17T04:32:20.054730+0000","last_clean":"2024-05-17T04:32:20.054730+0000","last_became_active":"2024-05-17T04:31:55.988366+0000","last_became_peered":"2024-05-17T04:31:55.988366+0000","last_unstale":"2024-05-17T04:32:20.054730+0000","last_undegraded":"2024-05-17T04:32:20.054730+0000","last_fullsized":"2024-05-17T04:32:20.054730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:36:30.684945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036229+0000","last_change":"2024-05-17T04:32:20.059700+0000","last_active":"2024-05-17T04:32:47.036229+0000","last_peered":"2024-05-17T04:32:47.036229+0000","last_clean":"2024-05-17T04:32:47.036229+0000","last_became_active":"2024-05-17T04:32:20.059234+0000","last_became_peered":"2024-05-17T04:32:20.059234+0000","last_unstale":"2024-05-17T04:32:47.036229+0000","last_undegraded":"2024-05-17T04:32:47.036229+0000","last_fullsized":"2024-05-17T04:32:47.036229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:25:47.214832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056753+0000","last_change":"2024-05-17T04:31:42.926078+0000","last_active":"2024-05-17T04:32:20.056753+0000","last_peered":"2024-05-17T04:32:20.056753+0000","last_clean":"2024-05-17T04:32:20.056753+0000","last_became_active":"2024-05-17T04:31:42.925755+0000","last_became_peered":"2024-05-17T04:31:42.925755+0000","last_unstale":"2024-05-17T04:32:20.056753+0000","last_undegraded":"2024-05-17T04:32:20.056753+0000","last_fullsized":"2024-05-17T04:32:20.056753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:30:16.952810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197720+0000","last_change":"2024-05-17T04:31:56.178973+0000","last_active":"2024-05-17T04:32:20.197720+0000","last_peered":"2024-05-17T04:32:20.197720+0000","last_clean":"2024-05-17T04:32:20.197720+0000","last_became_active":"2024-05-17T04:31:56.178359+0000","last_became_peered":"2024-05-17T04:31:56.178359+0000","last_unstale":"2024-05-17T04:32:20.197720+0000","last_undegraded":"2024-05-17T04:32:20.197720+0000","last_fullsized":"2024-05-17T04:32:20.197720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:22:48.921778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:09.942965+0000","last_change":"2024-05-17T04:31:39.887364+0000","last_active":"2024-05-17T04:33:09.942965+0000","last_peered":"2024-05-17T04:33:09.942965+0000","last_clean":"2024-05-17T04:33:09.942965+0000","last_became_active":"2024-05-17T04:31:39.886961+0000","last_became_peered":"2024-05-17T04:31:39.886961+0000","last_unstale":"2024-05-17T04:33:09.942965+0000","last_undegraded":"2024-05-17T04:33:09.942965+0000","last_fullsized":"2024-05-17T04:33:09.942965+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:48:20.417903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198520+0000","last_change":"2024-05-17T04:31:42.933338+0000","last_active":"2024-05-17T04:32:20.198520+0000","last_peered":"2024-05-17T04:32:20.198520+0000","last_clean":"2024-05-17T04:32:20.198520+0000","last_became_active":"2024-05-17T04:31:42.931619+0000","last_became_peered":"2024-05-17T04:31:42.931619+0000","last_unstale":"2024-05-17T04:32:20.198520+0000","last_undegraded":"2024-05-17T04:32:20.198520+0000","last_fullsized":"2024-05-17T04:32:20.198520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:52.565132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056879+0000","last_change":"2024-05-17T04:31:55.990398+0000","last_active":"2024-05-17T04:32:20.056879+0000","last_peered":"2024-05-17T04:32:20.056879+0000","last_clean":"2024-05-17T04:32:20.056879+0000","last_became_active":"2024-05-17T04:31:55.989595+0000","last_became_peered":"2024-05-17T04:31:55.989595+0000","last_unstale":"2024-05-17T04:32:20.056879+0000","last_undegraded":"2024-05-17T04:32:20.056879+0000","last_fullsized":"2024-05-17T04:32:20.056879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:41:37.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.034597+0000","last_change":"2024-05-17T04:31:40.443503+0000","last_active":"2024-05-17T04:32:47.034597+0000","last_peered":"2024-05-17T04:32:47.034597+0000","last_clean":"2024-05-17T04:32:47.034597+0000","last_became_active":"2024-05-17T04:31:40.443094+0000","last_became_peered":"2024-05-17T04:31:40.443094+0000","last_unstale":"2024-05-17T04:32:47.034597+0000","last_undegraded":"2024-05-17T04:32:47.034597+0000","last_fullsized":"2024-05-17T04:32:47.034597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:03:26.837211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052311+0000","last_change":"2024-05-17T04:31:42.933715+0000","last_active":"2024-05-17T04:32:20.052311+0000","last_peered":"2024-05-17T04:32:20.052311+0000","last_clean":"2024-05-17T04:32:20.052311+0000","last_became_active":"2024-05-17T04:31:42.933360+0000","last_became_peered":"2024-05-17T04:31:42.933360+0000","last_unstale":"2024-05-17T04:32:20.052311+0000","last_undegraded":"2024-05-17T04:32:20.052311+0000","last_fullsized":"2024-05-17T04:32:20.052311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:02:30.568910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058554+0000","last_change":"2024-05-17T04:31:56.179224+0000","last_active":"2024-05-17T04:32:20.058554+0000","last_peered":"2024-05-17T04:32:20.058554+0000","last_clean":"2024-05-17T04:32:20.058554+0000","last_became_active":"2024-05-17T04:31:56.178458+0000","last_became_peered":"2024-05-17T04:31:56.178458+0000","last_unstale":"2024-05-17T04:32:20.058554+0000","last_undegraded":"2024-05-17T04:32:20.058554+0000","last_fullsized":"2024-05-17T04:32:20.058554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:34:51.303044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037558+0000","last_change":"2024-05-17T04:31:39.896901+0000","last_active":"2024-05-17T04:32:47.037558+0000","last_peered":"2024-05-17T04:32:47.037558+0000","last_clean":"2024-05-17T04:32:47.037558+0000","last_became_active":"2024-05-17T04:31:39.896580+0000","last_became_peered":"2024-05-17T04:31:39.896580+0000","last_unstale":"2024-05-17T04:32:47.037558+0000","last_undegraded":"2024-05-17T04:32:47.037558+0000","last_fullsized":"2024-05-17T04:32:47.037558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:26:24.317276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052350+0000","last_change":"2024-05-17T04:31:42.935437+0000","last_active":"2024-05-17T04:32:20.052350+0000","last_peered":"2024-05-17T04:32:20.052350+0000","last_clean":"2024-05-17T04:32:20.052350+0000","last_became_active":"2024-05-17T04:31:42.935067+0000","last_became_peered":"2024-05-17T04:31:42.935067+0000","last_unstale":"2024-05-17T04:32:20.052350+0000","last_undegraded":"2024-05-17T04:32:20.052350+0000","last_fullsized":"2024-05-17T04:32:20.052350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:56:20.766093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055636+0000","last_change":"2024-05-17T04:31:55.989080+0000","last_active":"2024-05-17T04:32:20.055636+0000","last_peered":"2024-05-17T04:32:20.055636+0000","last_clean":"2024-05-17T04:32:20.055636+0000","last_became_active":"2024-05-17T04:31:55.988657+0000","last_became_peered":"2024-05-17T04:31:55.988657+0000","last_unstale":"2024-05-17T04:32:20.055636+0000","last_undegraded":"2024-05-17T04:32:20.055636+0000","last_fullsized":"2024-05-17T04:32:20.055636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:42:06.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.040252+0000","last_change":"2024-05-17T04:31:39.893877+0000","last_active":"2024-05-17T04:32:47.040252+0000","last_peered":"2024-05-17T04:32:47.040252+0000","last_clean":"2024-05-17T04:32:47.040252+0000","last_became_active":"2024-05-17T04:31:39.893479+0000","last_became_peered":"2024-05-17T04:31:39.893479+0000","last_unstale":"2024-05-17T04:32:47.040252+0000","last_undegraded":"2024-05-17T04:32:47.040252+0000","last_fullsized":"2024-05-17T04:32:47.040252+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:46:21.163132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054661+0000","last_change":"2024-05-17T04:31:42.934449+0000","last_active":"2024-05-17T04:32:20.054661+0000","last_peered":"2024-05-17T04:32:20.054661+0000","last_clean":"2024-05-17T04:32:20.054661+0000","last_became_active":"2024-05-17T04:31:42.934054+0000","last_became_peered":"2024-05-17T04:31:42.934054+0000","last_unstale":"2024-05-17T04:32:20.054661+0000","last_undegraded":"2024-05-17T04:32:20.054661+0000","last_fullsized":"2024-05-17T04:32:20.054661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:35.344333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059309+0000","last_change":"2024-05-17T04:31:55.986935+0000","last_active":"2024-05-17T04:32:20.059309+0000","last_peered":"2024-05-17T04:32:20.059309+0000","last_clean":"2024-05-17T04:32:20.059309+0000","last_became_active":"2024-05-17T04:31:55.986471+0000","last_became_peered":"2024-05-17T04:31:55.986471+0000","last_unstale":"2024-05-17T04:32:20.059309+0000","last_undegraded":"2024-05-17T04:32:20.059309+0000","last_fullsized":"2024-05-17T04:32:20.059309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:04:09.479135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057278+0000","last_change":"2024-05-17T04:31:39.890000+0000","last_active":"2024-05-17T04:32:20.057278+0000","last_peered":"2024-05-17T04:32:20.057278+0000","last_clean":"2024-05-17T04:32:20.057278+0000","last_became_active":"2024-05-17T04:31:39.887979+0000","last_became_peered":"2024-05-17T04:31:39.887979+0000","last_unstale":"2024-05-17T04:32:20.057278+0000","last_undegraded":"2024-05-17T04:32:20.057278+0000","last_fullsized":"2024-05-17T04:32:20.057278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:44:13.580387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198348+0000","last_change":"2024-05-17T04:31:42.943980+0000","last_active":"2024-05-17T04:32:20.198348+0000","last_peered":"2024-05-17T04:32:20.198348+0000","last_clean":"2024-05-17T04:32:20.198348+0000","last_became_active":"2024-05-17T04:31:42.943647+0000","last_became_peered":"2024-05-17T04:31:42.943647+0000","last_unstale":"2024-05-17T04:32:20.198348+0000","last_undegraded":"2024-05-17T04:32:20.198348+0000","last_fullsized":"2024-05-17T04:32:20.198348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:17:15.909198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060156+0000","last_change":"2024-05-17T04:31:56.178896+0000","last_active":"2024-05-17T04:32:20.060156+0000","last_peered":"2024-05-17T04:32:20.060156+0000","last_clean":"2024-05-17T04:32:20.060156+0000","last_became_active":"2024-05-17T04:31:56.178298+0000","last_became_peered":"2024-05-17T04:31:56.178298+0000","last_unstale":"2024-05-17T04:32:20.060156+0000","last_undegraded":"2024-05-17T04:32:20.060156+0000","last_fullsized":"2024-05-17T04:32:20.060156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:30:32.759024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037577+0000","last_change":"2024-05-17T04:31:39.894705+0000","last_active":"2024-05-17T04:32:47.037577+0000","last_peered":"2024-05-17T04:32:47.037577+0000","last_clean":"2024-05-17T04:32:47.037577+0000","last_became_active":"2024-05-17T04:31:39.894312+0000","last_became_peered":"2024-05-17T04:31:39.894312+0000","last_unstale":"2024-05-17T04:32:47.037577+0000","last_undegraded":"2024-05-17T04:32:47.037577+0000","last_fullsized":"2024-05-17T04:32:47.037577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:41:13.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060452+0000","last_change":"2024-05-17T04:31:42.941171+0000","last_active":"2024-05-17T04:32:20.060452+0000","last_peered":"2024-05-17T04:32:20.060452+0000","last_clean":"2024-05-17T04:32:20.060452+0000","last_became_active":"2024-05-17T04:31:42.940724+0000","last_became_peered":"2024-05-17T04:31:42.940724+0000","last_unstale":"2024-05-17T04:32:20.060452+0000","last_undegraded":"2024-05-17T04:32:20.060452+0000","last_fullsized":"2024-05-17T04:32:20.060452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:26:08.933000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054328+0000","last_change":"2024-05-17T04:31:55.992427+0000","last_active":"2024-05-17T04:32:20.054328+0000","last_peered":"2024-05-17T04:32:20.054328+0000","last_clean":"2024-05-17T04:32:20.054328+0000","last_became_active":"2024-05-17T04:31:55.991936+0000","last_became_peered":"2024-05-17T04:31:55.991936+0000","last_unstale":"2024-05-17T04:32:20.054328+0000","last_undegraded":"2024-05-17T04:32:20.054328+0000","last_fullsized":"2024-05-17T04:32:20.054328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:30:01.610347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.042774+0000","last_change":"2024-05-17T04:32:20.062349+0000","last_active":"2024-05-17T04:32:48.042774+0000","last_peered":"2024-05-17T04:32:48.042774+0000","last_clean":"2024-05-17T04:32:48.042774+0000","last_became_active":"2024-05-17T04:32:20.061708+0000","last_became_peered":"2024-05-17T04:32:20.061708+0000","last_unstale":"2024-05-17T04:32:48.042774+0000","last_undegraded":"2024-05-17T04:32:48.042774+0000","last_fullsized":"2024-05-17T04:32:48.042774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:51:02.946670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060650+0000","last_change":"2024-05-17T04:31:42.931998+0000","last_active":"2024-05-17T04:32:20.060650+0000","last_peered":"2024-05-17T04:32:20.060650+0000","last_clean":"2024-05-17T04:32:20.060650+0000","last_became_active":"2024-05-17T04:31:42.929898+0000","last_became_peered":"2024-05-17T04:31:42.929898+0000","last_unstale":"2024-05-17T04:32:20.060650+0000","last_undegraded":"2024-05-17T04:32:20.060650+0000","last_fullsized":"2024-05-17T04:32:20.060650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:55:35.349637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.051781+0000","last_change":"2024-05-17T04:31:55.990093+0000","last_active":"2024-05-17T04:32:20.051781+0000","last_peered":"2024-05-17T04:32:20.051781+0000","last_clean":"2024-05-17T04:32:20.051781+0000","last_became_active":"2024-05-17T04:31:55.989665+0000","last_became_peered":"2024-05-17T04:31:55.989665+0000","last_unstale":"2024-05-17T04:32:20.051781+0000","last_undegraded":"2024-05-17T04:32:20.051781+0000","last_fullsized":"2024-05-17T04:32:20.051781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:37:43.727322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060975+0000","last_change":"2024-05-17T04:31:39.893601+0000","last_active":"2024-05-17T04:32:20.060975+0000","last_peered":"2024-05-17T04:32:20.060975+0000","last_clean":"2024-05-17T04:32:20.060975+0000","last_became_active":"2024-05-17T04:31:39.893141+0000","last_became_peered":"2024-05-17T04:31:39.893141+0000","last_unstale":"2024-05-17T04:32:20.060975+0000","last_undegraded":"2024-05-17T04:32:20.060975+0000","last_fullsized":"2024-05-17T04:32:20.060975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:05:09.173607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060155+0000","last_change":"2024-05-17T04:31:42.932083+0000","last_active":"2024-05-17T04:32:20.060155+0000","last_peered":"2024-05-17T04:32:20.060155+0000","last_clean":"2024-05-17T04:32:20.060155+0000","last_became_active":"2024-05-17T04:31:42.931520+0000","last_became_peered":"2024-05-17T04:31:42.931520+0000","last_unstale":"2024-05-17T04:32:20.060155+0000","last_undegraded":"2024-05-17T04:32:20.060155+0000","last_fullsized":"2024-05-17T04:32:20.060155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:47:55.057982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059905+0000","last_change":"2024-05-17T04:31:55.990315+0000","last_active":"2024-05-17T04:32:20.059905+0000","last_peered":"2024-05-17T04:32:20.059905+0000","last_clean":"2024-05-17T04:32:20.059905+0000","last_became_active":"2024-05-17T04:31:55.989565+0000","last_became_peered":"2024-05-17T04:31:55.989565+0000","last_unstale":"2024-05-17T04:32:20.059905+0000","last_undegraded":"2024-05-17T04:32:20.059905+0000","last_fullsized":"2024-05-17T04:32:20.059905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:03.699211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056710+0000","last_change":"2024-05-17T04:31:39.890621+0000","last_active":"2024-05-17T04:32:20.056710+0000","last_peered":"2024-05-17T04:32:20.056710+0000","last_clean":"2024-05-17T04:32:20.056710+0000","last_became_active":"2024-05-17T04:31:39.890180+0000","last_became_peered":"2024-05-17T04:31:39.890180+0000","last_unstale":"2024-05-17T04:32:20.056710+0000","last_undegraded":"2024-05-17T04:32:20.056710+0000","last_fullsized":"2024-05-17T04:32:20.056710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:42.018482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060778+0000","last_change":"2024-05-17T04:31:42.928951+0000","last_active":"2024-05-17T04:32:20.060778+0000","last_peered":"2024-05-17T04:32:20.060778+0000","last_clean":"2024-05-17T04:32:20.060778+0000","last_became_active":"2024-05-17T04:31:42.928382+0000","last_became_peered":"2024-05-17T04:31:42.928382+0000","last_unstale":"2024-05-17T04:32:20.060778+0000","last_undegraded":"2024-05-17T04:32:20.060778+0000","last_fullsized":"2024-05-17T04:32:20.060778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:36.307351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054747+0000","last_change":"2024-05-17T04:31:42.926377+0000","last_active":"2024-05-17T04:32:20.054747+0000","last_peered":"2024-05-17T04:32:20.054747+0000","last_clean":"2024-05-17T04:32:20.054747+0000","last_became_active":"2024-05-17T04:31:42.926007+0000","last_became_peered":"2024-05-17T04:31:42.926007+0000","last_unstale":"2024-05-17T04:32:20.054747+0000","last_undegraded":"2024-05-17T04:32:20.054747+0000","last_fullsized":"2024-05-17T04:32:20.054747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:43:45.456144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057772+0000","last_change":"2024-05-17T04:31:42.932742+0000","last_active":"2024-05-17T04:32:20.057772+0000","last_peered":"2024-05-17T04:32:20.057772+0000","last_clean":"2024-05-17T04:32:20.057772+0000","last_became_active":"2024-05-17T04:31:42.932374+0000","last_became_peered":"2024-05-17T04:31:42.932374+0000","last_unstale":"2024-05-17T04:32:20.057772+0000","last_undegraded":"2024-05-17T04:32:20.057772+0000","last_fullsized":"2024-05-17T04:32:20.057772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:37.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057008+0000","last_change":"2024-05-17T04:31:42.940870+0000","last_active":"2024-05-17T04:32:20.057008+0000","last_peered":"2024-05-17T04:32:20.057008+0000","last_clean":"2024-05-17T04:32:20.057008+0000","last_became_active":"2024-05-17T04:31:42.940463+0000","last_became_peered":"2024-05-17T04:31:42.940463+0000","last_unstale":"2024-05-17T04:32:20.057008+0000","last_undegraded":"2024-05-17T04:32:20.057008+0000","last_fullsized":"2024-05-17T04:32:20.057008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:48:52.684601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052000+0000","last_change":"2024-05-17T04:31:42.936055+0000","last_active":"2024-05-17T04:32:20.052000+0000","last_peered":"2024-05-17T04:32:20.052000+0000","last_clean":"2024-05-17T04:32:20.052000+0000","last_became_active":"2024-05-17T04:31:42.934013+0000","last_became_peered":"2024-05-17T04:31:42.934013+0000","last_unstale":"2024-05-17T04:32:20.052000+0000","last_undegraded":"2024-05-17T04:32:20.052000+0000","last_fullsized":"2024-05-17T04:32:20.052000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:03:10.502727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056778+0000","last_change":"2024-05-17T04:31:42.942930+0000","last_active":"2024-05-17T04:32:20.056778+0000","last_peered":"2024-05-17T04:32:20.056778+0000","last_clean":"2024-05-17T04:32:20.056778+0000","last_became_active":"2024-05-17T04:31:42.942585+0000","last_became_peered":"2024-05-17T04:31:42.942585+0000","last_unstale":"2024-05-17T04:32:20.056778+0000","last_undegraded":"2024-05-17T04:32:20.056778+0000","last_fullsized":"2024-05-17T04:32:20.056778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:28:45.780505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056274+0000","last_change":"2024-05-17T04:31:42.923752+0000","last_active":"2024-05-17T04:32:20.056274+0000","last_peered":"2024-05-17T04:32:20.056274+0000","last_clean":"2024-05-17T04:32:20.056274+0000","last_became_active":"2024-05-17T04:31:42.923375+0000","last_became_peered":"2024-05-17T04:31:42.923375+0000","last_unstale":"2024-05-17T04:32:20.056274+0000","last_undegraded":"2024-05-17T04:32:20.056274+0000","last_fullsized":"2024-05-17T04:32:20.056274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:32:13.755314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057926+0000","last_change":"2024-05-17T04:31:42.928666+0000","last_active":"2024-05-17T04:32:20.057926+0000","last_peered":"2024-05-17T04:32:20.057926+0000","last_clean":"2024-05-17T04:32:20.057926+0000","last_became_active":"2024-05-17T04:31:42.928291+0000","last_became_peered":"2024-05-17T04:31:42.928291+0000","last_unstale":"2024-05-17T04:32:20.057926+0000","last_undegraded":"2024-05-17T04:32:20.057926+0000","last_fullsized":"2024-05-17T04:32:20.057926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:12:32.253374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198633+0000","last_change":"2024-05-17T04:31:42.926316+0000","last_active":"2024-05-17T04:32:20.198633+0000","last_peered":"2024-05-17T04:32:20.198633+0000","last_clean":"2024-05-17T04:32:20.198633+0000","last_became_active":"2024-05-17T04:31:42.925823+0000","last_became_peered":"2024-05-17T04:31:42.925823+0000","last_unstale":"2024-05-17T04:32:20.198633+0000","last_undegraded":"2024-05-17T04:32:20.198633+0000","last_fullsized":"2024-05-17T04:32:20.198633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:16:15.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061004+0000","last_change":"2024-05-17T04:31:42.945587+0000","last_active":"2024-05-17T04:32:20.061004+0000","last_peered":"2024-05-17T04:32:20.061004+0000","last_clean":"2024-05-17T04:32:20.061004+0000","last_became_active":"2024-05-17T04:31:42.945238+0000","last_became_peered":"2024-05-17T04:31:42.945238+0000","last_unstale":"2024-05-17T04:32:20.061004+0000","last_undegraded":"2024-05-17T04:32:20.061004+0000","last_fullsized":"2024-05-17T04:32:20.061004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:17:29.584023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198069+0000","last_change":"2024-05-17T04:31:42.925883+0000","last_active":"2024-05-17T04:32:20.198069+0000","last_peered":"2024-05-17T04:32:20.198069+0000","last_clean":"2024-05-17T04:32:20.198069+0000","last_became_active":"2024-05-17T04:31:42.925506+0000","last_became_peered":"2024-05-17T04:31:42.925506+0000","last_unstale":"2024-05-17T04:32:20.198069+0000","last_undegraded":"2024-05-17T04:32:20.198069+0000","last_fullsized":"2024-05-17T04:32:20.198069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:31:16.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055198+0000","last_change":"2024-05-17T04:31:42.928444+0000","last_active":"2024-05-17T04:32:20.055198+0000","last_peered":"2024-05-17T04:32:20.055198+0000","last_clean":"2024-05-17T04:32:20.055198+0000","last_became_active":"2024-05-17T04:31:42.928060+0000","last_became_peered":"2024-05-17T04:31:42.928060+0000","last_unstale":"2024-05-17T04:32:20.055198+0000","last_undegraded":"2024-05-17T04:32:20.055198+0000","last_fullsized":"2024-05-17T04:32:20.055198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:47:29.714401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056251+0000","last_change":"2024-05-17T04:31:42.937010+0000","last_active":"2024-05-17T04:32:20.056251+0000","last_peered":"2024-05-17T04:32:20.056251+0000","last_clean":"2024-05-17T04:32:20.056251+0000","last_became_active":"2024-05-17T04:31:42.936675+0000","last_became_peered":"2024-05-17T04:31:42.936675+0000","last_unstale":"2024-05-17T04:32:20.056251+0000","last_undegraded":"2024-05-17T04:32:20.056251+0000","last_fullsized":"2024-05-17T04:32:20.056251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:06:22.421966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054911+0000","last_change":"2024-05-17T04:31:42.924943+0000","last_active":"2024-05-17T04:32:20.054911+0000","last_peered":"2024-05-17T04:32:20.054911+0000","last_clean":"2024-05-17T04:32:20.054911+0000","last_became_active":"2024-05-17T04:31:42.924578+0000","last_became_peered":"2024-05-17T04:31:42.924578+0000","last_unstale":"2024-05-17T04:32:20.054911+0000","last_undegraded":"2024-05-17T04:32:20.054911+0000","last_fullsized":"2024-05-17T04:32:20.054911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:22:58.614038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284318,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62199,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":178,"num_read_kb":156,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":186597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":496276,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906983,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1141698,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070509,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":500564,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":496344,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462967,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1092688,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":491644,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":549943,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":493032,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":550520,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":491838,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":496685,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":1087885,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T04:33:20.067 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T04:33:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:19 smithi105 ceph-mon[34795]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-17T04:33:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:19 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-17T04:33:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:19 smithi110 ceph-mon[35497]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-17T04:33:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:19 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-17T04:33:20.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:19 smithi050 ceph-mon[31739]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-17T04:33:20.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-05-17T04:33:20.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:19 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-17T04:33:20.330 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:21.181 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:20 smithi050 ceph-mon[31739]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:21.182 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-17T04:33:21.182 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:20 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-17T04:33:21.182 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:20 smithi050 ceph-mon[31739]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T04:33:21.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:20 smithi105 ceph-mon[34795]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:21.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-17T04:33:21.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:20 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-17T04:33:21.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:20 smithi105 ceph-mon[34795]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T04:33:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:20 smithi110 ceph-mon[35497]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-05-17T04:33:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:20 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-17T04:33:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:20 smithi110 ceph-mon[35497]: osdmap e88: 12 total, 12 up, 12 in 2024-05-17T04:33:22.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 -- 172.21.15.50:0/2698392170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 msgr2=0x7f223c104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 --2- 172.21.15.50:0/2698392170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c104320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b0a0 tx=0x7f222c00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 -- 172.21.15.50:0/2698392170 shutdown_connections 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 --2- 172.21.15.50:0/2698392170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 0x7f223c110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 --2- 172.21.15.50:0/2698392170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c104320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 --2- 172.21.15.50:0/2698392170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 0x7f223c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 -- 172.21.15.50:0/2698392170 >> 172.21.15.50:0/2698392170 conn(0x7f223c0ff9b0 msgr2=0x7f223c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 -- 172.21.15.50:0/2698392170 shutdown_connections 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.169+0000 7f2243307700 1 -- 172.21.15.50:0/2698392170 wait complete. 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.170+0000 7f2243307700 1 Processor -- start 2024-05-17T04:33:22.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.170+0000 7f2243307700 1 -- start start 2024-05-17T04:33:22.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:22.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 0x7f223c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:22.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 0x7f223c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:22.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113c90 con 0x7f223c103f10 2024-05-17T04:33:22.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113b10 con 0x7f223c108840 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f2243307700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f223c113e10 con 0x7f223c109190 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50034/0 (socket says 172.21.15.50:50034) 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 -- 172.21.15.50:0/873568791 learned_addr learned my addr 172.21.15.50:0/873568791 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22408a2700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 0x7f223c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22418a4700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 0x7f223c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 -- 172.21.15.50:0/873568791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 msgr2=0x7f223c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 0x7f223c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 -- 172.21.15.50:0/873568791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 msgr2=0x7f223c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 0x7f223c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.171+0000 7f22410a3700 1 -- 172.21.15.50:0/873568791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c19f580 con 0x7f223c103f10 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f22410a3700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f222c0062a0 tx=0x7f222c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:22.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c021070 con 0x7f223c103f10 2024-05-17T04:33:22.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c19f810 con 0x7f223c103f10 2024-05-17T04:33:22.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f222c007b80 con 0x7f223c103f10 2024-05-17T04:33:22.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0052a0 con 0x7f223c103f10 2024-05-17T04:33:22.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.172+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c19fcf0 con 0x7f223c103f10 2024-05-17T04:33:22.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.173+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223c104320 con 0x7f223c103f10 2024-05-17T04:33:22.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.174+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f222c024070 con 0x7f223c103f10 2024-05-17T04:33:22.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.175+0000 7f223a7fc700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 0x7f22240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:22.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.175+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f222c00f020 con 0x7f223c103f10 2024-05-17T04:33:22.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.175+0000 7f22408a2700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 0x7f22240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:22.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.176+0000 7f22408a2700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 0x7f22240b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f223c107420 tx=0x7f223000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:22.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.178+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f222c02b030 con 0x7f223c103f10 2024-05-17T04:33:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:21 smithi105 ceph-mon[34795]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:33:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:21 smithi105 ceph-mon[34795]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T04:33:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:21 smithi110 ceph-mon[35497]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:33:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:21 smithi110 ceph-mon[35497]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T04:33:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:21 smithi050 ceph-mon[31739]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:33:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:21 smithi050 ceph-mon[31739]: osdmap e89: 12 total, 12 up, 12 in 2024-05-17T04:33:22.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.392+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f223c19ba10 con 0x7f22240b2910 2024-05-17T04:33:22.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.398+0000 7f223a7fc700 1 -- 172.21.15.50:0/873568791 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340621 (secure 0 0 0) 0x7f223c19ba10 con 0x7f22240b2910 2024-05-17T04:33:22.402 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 msgr2=0x7f22240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 0x7f22240b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f223c107420 tx=0x7f223000b410 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 msgr2=0x7f223c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f222c0062a0 tx=0x7f222c004dd0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 shutdown_connections 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f223c109190 0x7f223c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22240b2910 0x7f22240b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f223c103f10 0x7f223c19a6b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 --2- 172.21.15.50:0/873568791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f223c108840 0x7f223c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 >> 172.21.15.50:0/873568791 conn(0x7f223c0ff9b0 msgr2=0x7f223c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 shutdown_connections 2024-05-17T04:33:22.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:22.403+0000 7f2243307700 1 -- 172.21.15.50:0/873568791 wait complete. 2024-05-17T04:33:22.406 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-17T04:33:23.217 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-05-17T04:33:20.930622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657066,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":280,"num_read_kb":235,"num_write":294,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373300,"kb_used_data":11668,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568524,"statfs":{"total":1151940427776,"available":1151558168576,"internally_reserved":0,"allocated":11948032,"data_stored":7897647,"data_compressed":425040,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":35,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.202654"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055501+0000","last_change":"2024-05-17T04:31:42.946840+0000","last_active":"2024-05-17T04:32:20.055501+0000","last_peered":"2024-05-17T04:32:20.055501+0000","last_clean":"2024-05-17T04:32:20.055501+0000","last_became_active":"2024-05-17T04:31:42.946391+0000","last_became_peered":"2024-05-17T04:31:42.946391+0000","last_unstale":"2024-05-17T04:32:20.055501+0000","last_undegraded":"2024-05-17T04:32:20.055501+0000","last_fullsized":"2024-05-17T04:32:20.055501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:06:29.922596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285327+0000","last_change":"2024-05-17T04:31:42.934805+0000","last_active":"2024-05-17T04:33:20.285327+0000","last_peered":"2024-05-17T04:33:20.285327+0000","last_clean":"2024-05-17T04:33:20.285327+0000","last_became_active":"2024-05-17T04:31:42.933527+0000","last_became_peered":"2024-05-17T04:31:42.933527+0000","last_unstale":"2024-05-17T04:33:20.285327+0000","last_undegraded":"2024-05-17T04:33:20.285327+0000","last_fullsized":"2024-05-17T04:33:20.285327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:31.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285167+0000","last_change":"2024-05-17T04:31:42.935619+0000","last_active":"2024-05-17T04:33:20.285167+0000","last_peered":"2024-05-17T04:33:20.285167+0000","last_clean":"2024-05-17T04:33:20.285167+0000","last_became_active":"2024-05-17T04:31:42.935270+0000","last_became_peered":"2024-05-17T04:31:42.935270+0000","last_unstale":"2024-05-17T04:33:20.285167+0000","last_undegraded":"2024-05-17T04:33:20.285167+0000","last_fullsized":"2024-05-17T04:33:20.285167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:17.059859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060975+0000","last_change":"2024-05-17T04:31:42.928754+0000","last_active":"2024-05-17T04:32:20.060975+0000","last_peered":"2024-05-17T04:32:20.060975+0000","last_clean":"2024-05-17T04:32:20.060975+0000","last_became_active":"2024-05-17T04:31:42.927407+0000","last_became_peered":"2024-05-17T04:31:42.927407+0000","last_unstale":"2024-05-17T04:32:20.060975+0000","last_undegraded":"2024-05-17T04:32:20.060975+0000","last_fullsized":"2024-05-17T04:32:20.060975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:38:33.270224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052458+0000","last_change":"2024-05-17T04:31:42.929405+0000","last_active":"2024-05-17T04:32:20.052458+0000","last_peered":"2024-05-17T04:32:20.052458+0000","last_clean":"2024-05-17T04:32:20.052458+0000","last_became_active":"2024-05-17T04:31:42.928970+0000","last_became_peered":"2024-05-17T04:31:42.928970+0000","last_unstale":"2024-05-17T04:32:20.052458+0000","last_undegraded":"2024-05-17T04:32:20.052458+0000","last_fullsized":"2024-05-17T04:32:20.052458+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:22:39.866681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056897+0000","last_change":"2024-05-17T04:31:42.929773+0000","last_active":"2024-05-17T04:32:20.056897+0000","last_peered":"2024-05-17T04:32:20.056897+0000","last_clean":"2024-05-17T04:32:20.056897+0000","last_became_active":"2024-05-17T04:31:42.929449+0000","last_became_peered":"2024-05-17T04:31:42.929449+0000","last_unstale":"2024-05-17T04:32:20.056897+0000","last_undegraded":"2024-05-17T04:32:20.056897+0000","last_fullsized":"2024-05-17T04:32:20.056897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:05:54.544961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059532+0000","last_change":"2024-05-17T04:31:42.930674+0000","last_active":"2024-05-17T04:32:20.059532+0000","last_peered":"2024-05-17T04:32:20.059532+0000","last_clean":"2024-05-17T04:32:20.059532+0000","last_became_active":"2024-05-17T04:31:42.929752+0000","last_became_peered":"2024-05-17T04:31:42.929752+0000","last_unstale":"2024-05-17T04:32:20.059532+0000","last_undegraded":"2024-05-17T04:32:20.059532+0000","last_fullsized":"2024-05-17T04:32:20.059532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:54:56.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197989+0000","last_change":"2024-05-17T04:31:42.941443+0000","last_active":"2024-05-17T04:32:20.197989+0000","last_peered":"2024-05-17T04:32:20.197989+0000","last_clean":"2024-05-17T04:32:20.197989+0000","last_became_active":"2024-05-17T04:31:42.941052+0000","last_became_peered":"2024-05-17T04:31:42.941052+0000","last_unstale":"2024-05-17T04:32:20.197989+0000","last_undegraded":"2024-05-17T04:32:20.197989+0000","last_fullsized":"2024-05-17T04:32:20.197989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:07.151040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054910+0000","last_change":"2024-05-17T04:31:42.938708+0000","last_active":"2024-05-17T04:32:20.054910+0000","last_peered":"2024-05-17T04:32:20.054910+0000","last_clean":"2024-05-17T04:32:20.054910+0000","last_became_active":"2024-05-17T04:31:42.938330+0000","last_became_peered":"2024-05-17T04:31:42.938330+0000","last_unstale":"2024-05-17T04:32:20.054910+0000","last_undegraded":"2024-05-17T04:32:20.054910+0000","last_fullsized":"2024-05-17T04:32:20.054910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:23.882565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056968+0000","last_change":"2024-05-17T04:31:42.932779+0000","last_active":"2024-05-17T04:32:20.056968+0000","last_peered":"2024-05-17T04:32:20.056968+0000","last_clean":"2024-05-17T04:32:20.056968+0000","last_became_active":"2024-05-17T04:31:42.932429+0000","last_became_peered":"2024-05-17T04:31:42.932429+0000","last_unstale":"2024-05-17T04:32:20.056968+0000","last_undegraded":"2024-05-17T04:32:20.056968+0000","last_fullsized":"2024-05-17T04:32:20.056968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:22:17.139775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060313+0000","last_change":"2024-05-17T04:31:42.932713+0000","last_active":"2024-05-17T04:32:20.060313+0000","last_peered":"2024-05-17T04:32:20.060313+0000","last_clean":"2024-05-17T04:32:20.060313+0000","last_became_active":"2024-05-17T04:31:42.931451+0000","last_became_peered":"2024-05-17T04:31:42.931451+0000","last_unstale":"2024-05-17T04:32:20.060313+0000","last_undegraded":"2024-05-17T04:32:20.060313+0000","last_fullsized":"2024-05-17T04:32:20.060313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:49:03.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057901+0000","last_change":"2024-05-17T04:32:20.057506+0000","last_active":"2024-05-17T04:32:20.057901+0000","last_peered":"2024-05-17T04:32:20.057901+0000","last_clean":"2024-05-17T04:32:20.057901+0000","last_became_active":"2024-05-17T04:32:20.057245+0000","last_became_peered":"2024-05-17T04:32:20.057245+0000","last_unstale":"2024-05-17T04:32:20.057901+0000","last_undegraded":"2024-05-17T04:32:20.057901+0000","last_fullsized":"2024-05-17T04:32:20.057901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:14.714699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056367+0000","last_change":"2024-05-17T04:31:42.930412+0000","last_active":"2024-05-17T04:32:20.056367+0000","last_peered":"2024-05-17T04:32:20.056367+0000","last_clean":"2024-05-17T04:32:20.056367+0000","last_became_active":"2024-05-17T04:31:42.929930+0000","last_became_peered":"2024-05-17T04:31:42.929930+0000","last_unstale":"2024-05-17T04:32:20.056367+0000","last_undegraded":"2024-05-17T04:32:20.056367+0000","last_fullsized":"2024-05-17T04:32:20.056367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:19:12.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285006+0000","last_change":"2024-05-17T04:31:42.932696+0000","last_active":"2024-05-17T04:33:20.285006+0000","last_peered":"2024-05-17T04:33:20.285006+0000","last_clean":"2024-05-17T04:33:20.285006+0000","last_became_active":"2024-05-17T04:31:42.932391+0000","last_became_peered":"2024-05-17T04:31:42.932391+0000","last_unstale":"2024-05-17T04:33:20.285006+0000","last_undegraded":"2024-05-17T04:33:20.285006+0000","last_fullsized":"2024-05-17T04:33:20.285006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:10:13.071352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.284904+0000","last_change":"2024-05-17T04:31:42.943331+0000","last_active":"2024-05-17T04:33:20.284904+0000","last_peered":"2024-05-17T04:33:20.284904+0000","last_clean":"2024-05-17T04:33:20.284904+0000","last_became_active":"2024-05-17T04:31:42.942963+0000","last_became_peered":"2024-05-17T04:31:42.942963+0000","last_unstale":"2024-05-17T04:33:20.284904+0000","last_undegraded":"2024-05-17T04:33:20.284904+0000","last_fullsized":"2024-05-17T04:33:20.284904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:27:33.614843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.053942+0000","last_change":"2024-05-17T04:31:42.935951+0000","last_active":"2024-05-17T04:32:20.053942+0000","last_peered":"2024-05-17T04:32:20.053942+0000","last_clean":"2024-05-17T04:32:20.053942+0000","last_became_active":"2024-05-17T04:31:42.935564+0000","last_became_peered":"2024-05-17T04:31:42.935564+0000","last_unstale":"2024-05-17T04:32:20.053942+0000","last_undegraded":"2024-05-17T04:32:20.053942+0000","last_fullsized":"2024-05-17T04:32:20.053942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:15:28.282780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055352+0000","last_change":"2024-05-17T04:31:42.946858+0000","last_active":"2024-05-17T04:32:20.055352+0000","last_peered":"2024-05-17T04:32:20.055352+0000","last_clean":"2024-05-17T04:32:20.055352+0000","last_became_active":"2024-05-17T04:31:42.946398+0000","last_became_peered":"2024-05-17T04:31:42.946398+0000","last_unstale":"2024-05-17T04:32:20.055352+0000","last_undegraded":"2024-05-17T04:32:20.055352+0000","last_fullsized":"2024-05-17T04:32:20.055352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:51:09.584967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055592+0000","last_change":"2024-05-17T04:31:42.934716+0000","last_active":"2024-05-17T04:32:20.055592+0000","last_peered":"2024-05-17T04:32:20.055592+0000","last_clean":"2024-05-17T04:32:20.055592+0000","last_became_active":"2024-05-17T04:31:42.934379+0000","last_became_peered":"2024-05-17T04:31:42.934379+0000","last_unstale":"2024-05-17T04:32:20.055592+0000","last_undegraded":"2024-05-17T04:32:20.055592+0000","last_fullsized":"2024-05-17T04:32:20.055592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:54:01.154473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057919+0000","last_change":"2024-05-17T04:31:42.926424+0000","last_active":"2024-05-17T04:32:20.057919+0000","last_peered":"2024-05-17T04:32:20.057919+0000","last_clean":"2024-05-17T04:32:20.057919+0000","last_became_active":"2024-05-17T04:31:42.925948+0000","last_became_peered":"2024-05-17T04:31:42.925948+0000","last_unstale":"2024-05-17T04:32:20.057919+0000","last_undegraded":"2024-05-17T04:32:20.057919+0000","last_fullsized":"2024-05-17T04:32:20.057919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:27:48.495395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057109+0000","last_change":"2024-05-17T04:31:55.992551+0000","last_active":"2024-05-17T04:32:20.057109+0000","last_peered":"2024-05-17T04:32:20.057109+0000","last_clean":"2024-05-17T04:32:20.057109+0000","last_became_active":"2024-05-17T04:31:55.992138+0000","last_became_peered":"2024-05-17T04:31:55.992138+0000","last_unstale":"2024-05-17T04:32:20.057109+0000","last_undegraded":"2024-05-17T04:32:20.057109+0000","last_fullsized":"2024-05-17T04:32:20.057109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:43:40.296492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.043461+0000","last_change":"2024-05-17T04:31:39.890303+0000","last_active":"2024-05-17T04:32:48.043461+0000","last_peered":"2024-05-17T04:32:48.043461+0000","last_clean":"2024-05-17T04:32:48.043461+0000","last_became_active":"2024-05-17T04:31:39.889970+0000","last_became_peered":"2024-05-17T04:31:39.889970+0000","last_unstale":"2024-05-17T04:32:48.043461+0000","last_undegraded":"2024-05-17T04:32:48.043461+0000","last_fullsized":"2024-05-17T04:32:48.043461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:29:33.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198394+0000","last_change":"2024-05-17T04:31:42.942203+0000","last_active":"2024-05-17T04:32:20.198394+0000","last_peered":"2024-05-17T04:32:20.198394+0000","last_clean":"2024-05-17T04:32:20.198394+0000","last_became_active":"2024-05-17T04:31:42.940563+0000","last_became_peered":"2024-05-17T04:31:42.940563+0000","last_unstale":"2024-05-17T04:32:20.198394+0000","last_undegraded":"2024-05-17T04:32:20.198394+0000","last_fullsized":"2024-05-17T04:32:20.198394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:19:10.813809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059148+0000","last_change":"2024-05-17T04:31:55.988355+0000","last_active":"2024-05-17T04:32:20.059148+0000","last_peered":"2024-05-17T04:32:20.059148+0000","last_clean":"2024-05-17T04:32:20.059148+0000","last_became_active":"2024-05-17T04:31:55.987874+0000","last_became_peered":"2024-05-17T04:31:55.987874+0000","last_unstale":"2024-05-17T04:32:20.059148+0000","last_undegraded":"2024-05-17T04:32:20.059148+0000","last_fullsized":"2024-05-17T04:32:20.059148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:13:30.650959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198554+0000","last_change":"2024-05-17T04:31:39.896754+0000","last_active":"2024-05-17T04:32:20.198554+0000","last_peered":"2024-05-17T04:32:20.198554+0000","last_clean":"2024-05-17T04:32:20.198554+0000","last_became_active":"2024-05-17T04:31:39.896369+0000","last_became_peered":"2024-05-17T04:31:39.896369+0000","last_unstale":"2024-05-17T04:32:20.198554+0000","last_undegraded":"2024-05-17T04:32:20.198554+0000","last_fullsized":"2024-05-17T04:32:20.198554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:33:03.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285485+0000","last_change":"2024-05-17T04:31:42.938556+0000","last_active":"2024-05-17T04:33:20.285485+0000","last_peered":"2024-05-17T04:33:20.285485+0000","last_clean":"2024-05-17T04:33:20.285485+0000","last_became_active":"2024-05-17T04:31:42.938228+0000","last_became_peered":"2024-05-17T04:31:42.938228+0000","last_unstale":"2024-05-17T04:33:20.285485+0000","last_undegraded":"2024-05-17T04:33:20.285485+0000","last_fullsized":"2024-05-17T04:33:20.285485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:18:32.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059222+0000","last_change":"2024-05-17T04:31:55.991620+0000","last_active":"2024-05-17T04:32:20.059222+0000","last_peered":"2024-05-17T04:32:20.059222+0000","last_clean":"2024-05-17T04:32:20.059222+0000","last_became_active":"2024-05-17T04:31:55.990762+0000","last_became_peered":"2024-05-17T04:31:55.990762+0000","last_unstale":"2024-05-17T04:32:20.059222+0000","last_undegraded":"2024-05-17T04:32:20.059222+0000","last_fullsized":"2024-05-17T04:32:20.059222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:22:49.033835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.041261+0000","last_change":"2024-05-17T04:32:20.067654+0000","last_active":"2024-05-17T04:32:48.041261+0000","last_peered":"2024-05-17T04:32:48.041261+0000","last_clean":"2024-05-17T04:32:48.041261+0000","last_became_active":"2024-05-17T04:32:20.062315+0000","last_became_peered":"2024-05-17T04:32:20.062315+0000","last_unstale":"2024-05-17T04:32:48.041261+0000","last_undegraded":"2024-05-17T04:32:48.041261+0000","last_fullsized":"2024-05-17T04:32:48.041261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:32:52.479838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054655+0000","last_change":"2024-05-17T04:31:42.938076+0000","last_active":"2024-05-17T04:32:20.054655+0000","last_peered":"2024-05-17T04:32:20.054655+0000","last_clean":"2024-05-17T04:32:20.054655+0000","last_became_active":"2024-05-17T04:31:42.937177+0000","last_became_peered":"2024-05-17T04:31:42.937177+0000","last_unstale":"2024-05-17T04:32:20.054655+0000","last_undegraded":"2024-05-17T04:32:20.054655+0000","last_fullsized":"2024-05-17T04:32:20.054655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:08:29.759076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058302+0000","last_change":"2024-05-17T04:31:55.988774+0000","last_active":"2024-05-17T04:32:20.058302+0000","last_peered":"2024-05-17T04:32:20.058302+0000","last_clean":"2024-05-17T04:32:20.058302+0000","last_became_active":"2024-05-17T04:31:55.987934+0000","last_became_peered":"2024-05-17T04:31:55.987934+0000","last_unstale":"2024-05-17T04:32:20.058302+0000","last_undegraded":"2024-05-17T04:32:20.058302+0000","last_fullsized":"2024-05-17T04:32:20.058302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:24:53.408890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.053099+0000","last_change":"2024-05-17T04:31:39.896382+0000","last_active":"2024-05-17T04:32:20.053099+0000","last_peered":"2024-05-17T04:32:20.053099+0000","last_clean":"2024-05-17T04:32:20.053099+0000","last_became_active":"2024-05-17T04:31:39.895863+0000","last_became_peered":"2024-05-17T04:31:39.895863+0000","last_unstale":"2024-05-17T04:32:20.053099+0000","last_undegraded":"2024-05-17T04:32:20.053099+0000","last_fullsized":"2024-05-17T04:32:20.053099+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:08:37.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060759+0000","last_change":"2024-05-17T04:31:42.938624+0000","last_active":"2024-05-17T04:32:20.060759+0000","last_peered":"2024-05-17T04:32:20.060759+0000","last_clean":"2024-05-17T04:32:20.060759+0000","last_became_active":"2024-05-17T04:31:42.938197+0000","last_became_peered":"2024-05-17T04:31:42.938197+0000","last_unstale":"2024-05-17T04:32:20.060759+0000","last_undegraded":"2024-05-17T04:32:20.060759+0000","last_fullsized":"2024-05-17T04:32:20.060759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:47:14.474259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057672+0000","last_change":"2024-05-17T04:31:55.988254+0000","last_active":"2024-05-17T04:32:20.057672+0000","last_peered":"2024-05-17T04:32:20.057672+0000","last_clean":"2024-05-17T04:32:20.057672+0000","last_became_active":"2024-05-17T04:31:55.987790+0000","last_became_peered":"2024-05-17T04:31:55.987790+0000","last_unstale":"2024-05-17T04:32:20.057672+0000","last_undegraded":"2024-05-17T04:32:20.057672+0000","last_fullsized":"2024-05-17T04:32:20.057672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:07:03.189403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":78,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:17.097931+0000","last_change":"2024-05-17T04:31:39.894789+0000","last_active":"2024-05-17T04:33:17.097931+0000","last_peered":"2024-05-17T04:33:17.097931+0000","last_clean":"2024-05-17T04:33:17.097931+0000","last_became_active":"2024-05-17T04:31:39.894404+0000","last_became_peered":"2024-05-17T04:31:39.894404+0000","last_unstale":"2024-05-17T04:33:17.097931+0000","last_undegraded":"2024-05-17T04:33:17.097931+0000","last_fullsized":"2024-05-17T04:33:17.097931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:00:25.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052446+0000","last_change":"2024-05-17T04:31:42.932430+0000","last_active":"2024-05-17T04:32:20.052446+0000","last_peered":"2024-05-17T04:32:20.052446+0000","last_clean":"2024-05-17T04:32:20.052446+0000","last_became_active":"2024-05-17T04:31:42.932039+0000","last_became_peered":"2024-05-17T04:31:42.932039+0000","last_unstale":"2024-05-17T04:32:20.052446+0000","last_undegraded":"2024-05-17T04:32:20.052446+0000","last_fullsized":"2024-05-17T04:32:20.052446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:46:53.928092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057284+0000","last_change":"2024-05-17T04:31:55.992898+0000","last_active":"2024-05-17T04:32:20.057284+0000","last_peered":"2024-05-17T04:32:20.057284+0000","last_clean":"2024-05-17T04:32:20.057284+0000","last_became_active":"2024-05-17T04:31:55.992459+0000","last_became_peered":"2024-05-17T04:31:55.992459+0000","last_unstale":"2024-05-17T04:32:20.057284+0000","last_undegraded":"2024-05-17T04:32:20.057284+0000","last_fullsized":"2024-05-17T04:32:20.057284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:45:24.888533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.040360+0000","last_change":"2024-05-17T04:31:39.893014+0000","last_active":"2024-05-17T04:32:48.040360+0000","last_peered":"2024-05-17T04:32:48.040360+0000","last_clean":"2024-05-17T04:32:48.040360+0000","last_became_active":"2024-05-17T04:31:39.892602+0000","last_became_peered":"2024-05-17T04:31:39.892602+0000","last_unstale":"2024-05-17T04:32:48.040360+0000","last_undegraded":"2024-05-17T04:32:48.040360+0000","last_fullsized":"2024-05-17T04:32:48.040360+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:45:12.742516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061404+0000","last_change":"2024-05-17T04:31:42.945750+0000","last_active":"2024-05-17T04:32:20.061404+0000","last_peered":"2024-05-17T04:32:20.061404+0000","last_clean":"2024-05-17T04:32:20.061404+0000","last_became_active":"2024-05-17T04:31:42.945332+0000","last_became_peered":"2024-05-17T04:31:42.945332+0000","last_unstale":"2024-05-17T04:32:20.061404+0000","last_undegraded":"2024-05-17T04:32:20.061404+0000","last_fullsized":"2024-05-17T04:32:20.061404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:36:00.521489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055211+0000","last_change":"2024-05-17T04:31:55.984513+0000","last_active":"2024-05-17T04:32:20.055211+0000","last_peered":"2024-05-17T04:32:20.055211+0000","last_clean":"2024-05-17T04:32:20.055211+0000","last_became_active":"2024-05-17T04:31:55.984065+0000","last_became_peered":"2024-05-17T04:31:55.984065+0000","last_unstale":"2024-05-17T04:32:20.055211+0000","last_undegraded":"2024-05-17T04:32:20.055211+0000","last_fullsized":"2024-05-17T04:32:20.055211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:07:06.370969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.042200+0000","last_change":"2024-05-17T04:31:39.889551+0000","last_active":"2024-05-17T04:32:48.042200+0000","last_peered":"2024-05-17T04:32:48.042200+0000","last_clean":"2024-05-17T04:32:48.042200+0000","last_became_active":"2024-05-17T04:31:39.889150+0000","last_became_peered":"2024-05-17T04:31:39.889150+0000","last_unstale":"2024-05-17T04:32:48.042200+0000","last_undegraded":"2024-05-17T04:32:48.042200+0000","last_fullsized":"2024-05-17T04:32:48.042200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:14:06.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198119+0000","last_change":"2024-05-17T04:31:42.922236+0000","last_active":"2024-05-17T04:32:20.198119+0000","last_peered":"2024-05-17T04:32:20.198119+0000","last_clean":"2024-05-17T04:32:20.198119+0000","last_became_active":"2024-05-17T04:31:42.921907+0000","last_became_peered":"2024-05-17T04:31:42.921907+0000","last_unstale":"2024-05-17T04:32:20.198119+0000","last_undegraded":"2024-05-17T04:32:20.198119+0000","last_fullsized":"2024-05-17T04:32:20.198119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:10:09.941419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285259+0000","last_change":"2024-05-17T04:31:55.997210+0000","last_active":"2024-05-17T04:33:20.285259+0000","last_peered":"2024-05-17T04:33:20.285259+0000","last_clean":"2024-05-17T04:33:20.285259+0000","last_became_active":"2024-05-17T04:31:55.996795+0000","last_became_peered":"2024-05-17T04:31:55.996795+0000","last_unstale":"2024-05-17T04:33:20.285259+0000","last_undegraded":"2024-05-17T04:33:20.285259+0000","last_fullsized":"2024-05-17T04:33:20.285259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:51:42.271149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060686+0000","last_change":"2024-05-17T04:31:39.891980+0000","last_active":"2024-05-17T04:32:20.060686+0000","last_peered":"2024-05-17T04:32:20.060686+0000","last_clean":"2024-05-17T04:32:20.060686+0000","last_became_active":"2024-05-17T04:31:39.891636+0000","last_became_peered":"2024-05-17T04:31:39.891636+0000","last_unstale":"2024-05-17T04:32:20.060686+0000","last_undegraded":"2024-05-17T04:32:20.060686+0000","last_fullsized":"2024-05-17T04:32:20.060686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:05:33.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055092+0000","last_change":"2024-05-17T04:31:42.929970+0000","last_active":"2024-05-17T04:32:20.055092+0000","last_peered":"2024-05-17T04:32:20.055092+0000","last_clean":"2024-05-17T04:32:20.055092+0000","last_became_active":"2024-05-17T04:31:42.929587+0000","last_became_peered":"2024-05-17T04:31:42.929587+0000","last_unstale":"2024-05-17T04:32:20.055092+0000","last_undegraded":"2024-05-17T04:32:20.055092+0000","last_fullsized":"2024-05-17T04:32:20.055092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:05:38.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197931+0000","last_change":"2024-05-17T04:31:55.991388+0000","last_active":"2024-05-17T04:32:20.197931+0000","last_peered":"2024-05-17T04:32:20.197931+0000","last_clean":"2024-05-17T04:32:20.197931+0000","last_became_active":"2024-05-17T04:31:55.990979+0000","last_became_peered":"2024-05-17T04:31:55.990979+0000","last_unstale":"2024-05-17T04:32:20.197931+0000","last_undegraded":"2024-05-17T04:32:20.197931+0000","last_fullsized":"2024-05-17T04:32:20.197931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:34.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:18.098691+0000","last_change":"2024-05-17T04:31:39.893076+0000","last_active":"2024-05-17T04:33:18.098691+0000","last_peered":"2024-05-17T04:33:18.098691+0000","last_clean":"2024-05-17T04:33:18.098691+0000","last_became_active":"2024-05-17T04:31:39.892683+0000","last_became_peered":"2024-05-17T04:31:39.892683+0000","last_unstale":"2024-05-17T04:33:18.098691+0000","last_undegraded":"2024-05-17T04:33:18.098691+0000","last_fullsized":"2024-05-17T04:33:18.098691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:32:19.583662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054806+0000","last_change":"2024-05-17T04:31:42.938431+0000","last_active":"2024-05-17T04:32:20.054806+0000","last_peered":"2024-05-17T04:32:20.054806+0000","last_clean":"2024-05-17T04:32:20.054806+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:32:20.054806+0000","last_undegraded":"2024-05-17T04:32:20.054806+0000","last_fullsized":"2024-05-17T04:32:20.054806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:56:34.505923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.066417+0000","last_change":"2024-05-17T04:32:20.065723+0000","last_active":"2024-05-17T04:32:20.066417+0000","last_peered":"2024-05-17T04:32:20.066417+0000","last_clean":"2024-05-17T04:32:20.066417+0000","last_became_active":"2024-05-17T04:32:20.065266+0000","last_became_peered":"2024-05-17T04:32:20.065266+0000","last_unstale":"2024-05-17T04:32:20.066417+0000","last_undegraded":"2024-05-17T04:32:20.066417+0000","last_fullsized":"2024-05-17T04:32:20.066417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:16:54.256206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036246+0000","last_change":"2024-05-17T04:31:39.892324+0000","last_active":"2024-05-17T04:32:47.036246+0000","last_peered":"2024-05-17T04:32:47.036246+0000","last_clean":"2024-05-17T04:32:47.036246+0000","last_became_active":"2024-05-17T04:31:39.891398+0000","last_became_peered":"2024-05-17T04:31:39.891398+0000","last_unstale":"2024-05-17T04:32:47.036246+0000","last_undegraded":"2024-05-17T04:32:47.036246+0000","last_fullsized":"2024-05-17T04:32:47.036246+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:06.423894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285437+0000","last_change":"2024-05-17T04:31:42.948320+0000","last_active":"2024-05-17T04:33:20.285437+0000","last_peered":"2024-05-17T04:33:20.285437+0000","last_clean":"2024-05-17T04:33:20.285437+0000","last_became_active":"2024-05-17T04:31:42.948009+0000","last_became_peered":"2024-05-17T04:31:42.948009+0000","last_unstale":"2024-05-17T04:33:20.285437+0000","last_undegraded":"2024-05-17T04:33:20.285437+0000","last_fullsized":"2024-05-17T04:33:20.285437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:51:28.464925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059573+0000","last_change":"2024-05-17T04:31:55.984602+0000","last_active":"2024-05-17T04:32:20.059573+0000","last_peered":"2024-05-17T04:32:20.059573+0000","last_clean":"2024-05-17T04:32:20.059573+0000","last_became_active":"2024-05-17T04:31:55.984152+0000","last_became_peered":"2024-05-17T04:31:55.984152+0000","last_unstale":"2024-05-17T04:32:20.059573+0000","last_undegraded":"2024-05-17T04:32:20.059573+0000","last_fullsized":"2024-05-17T04:32:20.059573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:31:23.456878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036130+0000","last_change":"2024-05-17T04:31:39.896889+0000","last_active":"2024-05-17T04:32:47.036130+0000","last_peered":"2024-05-17T04:32:47.036130+0000","last_clean":"2024-05-17T04:32:47.036130+0000","last_became_active":"2024-05-17T04:31:39.896516+0000","last_became_peered":"2024-05-17T04:31:39.896516+0000","last_unstale":"2024-05-17T04:32:47.036130+0000","last_undegraded":"2024-05-17T04:32:47.036130+0000","last_fullsized":"2024-05-17T04:32:47.036130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:45:46.214589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054888+0000","last_change":"2024-05-17T04:31:42.928588+0000","last_active":"2024-05-17T04:32:20.054888+0000","last_peered":"2024-05-17T04:32:20.054888+0000","last_clean":"2024-05-17T04:32:20.054888+0000","last_became_active":"2024-05-17T04:31:42.928215+0000","last_became_peered":"2024-05-17T04:31:42.928215+0000","last_unstale":"2024-05-17T04:32:20.054888+0000","last_undegraded":"2024-05-17T04:32:20.054888+0000","last_fullsized":"2024-05-17T04:32:20.054888+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:49:31.894516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.063156+0000","last_change":"2024-05-17T04:32:20.063156+0000","last_active":"2024-05-17T04:32:20.063156+0000","last_peered":"2024-05-17T04:32:20.063156+0000","last_clean":"2024-05-17T04:32:20.063156+0000","last_became_active":"2024-05-17T04:32:20.062600+0000","last_became_peered":"2024-05-17T04:32:20.062600+0000","last_unstale":"2024-05-17T04:32:20.063156+0000","last_undegraded":"2024-05-17T04:32:20.063156+0000","last_fullsized":"2024-05-17T04:32:20.063156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:58:04.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:19.100389+0000","last_change":"2024-05-17T04:31:39.896849+0000","last_active":"2024-05-17T04:33:19.100389+0000","last_peered":"2024-05-17T04:33:19.100389+0000","last_clean":"2024-05-17T04:33:19.100389+0000","last_became_active":"2024-05-17T04:31:39.896134+0000","last_became_peered":"2024-05-17T04:31:39.896134+0000","last_unstale":"2024-05-17T04:33:19.100389+0000","last_undegraded":"2024-05-17T04:33:19.100389+0000","last_fullsized":"2024-05-17T04:33:19.100389+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:00:27.150148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":69,"num_read_kb":95,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052956+0000","last_change":"2024-05-17T04:31:42.932473+0000","last_active":"2024-05-17T04:32:20.052956+0000","last_peered":"2024-05-17T04:32:20.052956+0000","last_clean":"2024-05-17T04:32:20.052956+0000","last_became_active":"2024-05-17T04:31:42.932120+0000","last_became_peered":"2024-05-17T04:31:42.932120+0000","last_unstale":"2024-05-17T04:32:20.052956+0000","last_undegraded":"2024-05-17T04:32:20.052956+0000","last_fullsized":"2024-05-17T04:32:20.052956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:57:35.056071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061085+0000","last_change":"2024-05-17T04:31:55.999678+0000","last_active":"2024-05-17T04:32:20.061085+0000","last_peered":"2024-05-17T04:32:20.061085+0000","last_clean":"2024-05-17T04:32:20.061085+0000","last_became_active":"2024-05-17T04:31:55.999236+0000","last_became_peered":"2024-05-17T04:31:55.999236+0000","last_unstale":"2024-05-17T04:32:20.061085+0000","last_undegraded":"2024-05-17T04:32:20.061085+0000","last_fullsized":"2024-05-17T04:32:20.061085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:16:27.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.040339+0000","last_change":"2024-05-17T04:31:39.894334+0000","last_active":"2024-05-17T04:32:48.040339+0000","last_peered":"2024-05-17T04:32:48.040339+0000","last_clean":"2024-05-17T04:32:48.040339+0000","last_became_active":"2024-05-17T04:31:39.893801+0000","last_became_peered":"2024-05-17T04:31:39.893801+0000","last_unstale":"2024-05-17T04:32:48.040339+0000","last_undegraded":"2024-05-17T04:32:48.040339+0000","last_fullsized":"2024-05-17T04:32:48.040339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:26:07.143110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055984+0000","last_change":"2024-05-17T04:31:42.934598+0000","last_active":"2024-05-17T04:32:20.055984+0000","last_peered":"2024-05-17T04:32:20.055984+0000","last_clean":"2024-05-17T04:32:20.055984+0000","last_became_active":"2024-05-17T04:31:42.934268+0000","last_became_peered":"2024-05-17T04:31:42.934268+0000","last_unstale":"2024-05-17T04:32:20.055984+0000","last_undegraded":"2024-05-17T04:32:20.055984+0000","last_fullsized":"2024-05-17T04:32:20.055984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:50:13.868834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054952+0000","last_change":"2024-05-17T04:31:55.986888+0000","last_active":"2024-05-17T04:32:20.054952+0000","last_peered":"2024-05-17T04:32:20.054952+0000","last_clean":"2024-05-17T04:32:20.054952+0000","last_became_active":"2024-05-17T04:31:55.986369+0000","last_became_peered":"2024-05-17T04:31:55.986369+0000","last_unstale":"2024-05-17T04:32:20.054952+0000","last_undegraded":"2024-05-17T04:32:20.054952+0000","last_fullsized":"2024-05-17T04:32:20.054952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:42:40.672724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037676+0000","last_change":"2024-05-17T04:31:39.891226+0000","last_active":"2024-05-17T04:32:47.037676+0000","last_peered":"2024-05-17T04:32:47.037676+0000","last_clean":"2024-05-17T04:32:47.037676+0000","last_became_active":"2024-05-17T04:31:39.890376+0000","last_became_peered":"2024-05-17T04:31:39.890376+0000","last_unstale":"2024-05-17T04:32:47.037676+0000","last_undegraded":"2024-05-17T04:32:47.037676+0000","last_fullsized":"2024-05-17T04:32:47.037676+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:13:14.749546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.062998+0000","last_change":"2024-05-17T04:32:20.062998+0000","last_active":"2024-05-17T04:32:20.062998+0000","last_peered":"2024-05-17T04:32:20.062998+0000","last_clean":"2024-05-17T04:32:20.062998+0000","last_became_active":"2024-05-17T04:32:20.062485+0000","last_became_peered":"2024-05-17T04:32:20.062485+0000","last_unstale":"2024-05-17T04:32:20.062998+0000","last_undegraded":"2024-05-17T04:32:20.062998+0000","last_fullsized":"2024-05-17T04:32:20.062998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:20:23.822735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055417+0000","last_change":"2024-05-17T04:31:55.994500+0000","last_active":"2024-05-17T04:32:20.055417+0000","last_peered":"2024-05-17T04:32:20.055417+0000","last_clean":"2024-05-17T04:32:20.055417+0000","last_became_active":"2024-05-17T04:31:55.994130+0000","last_became_peered":"2024-05-17T04:31:55.994130+0000","last_unstale":"2024-05-17T04:32:20.055417+0000","last_undegraded":"2024-05-17T04:32:20.055417+0000","last_fullsized":"2024-05-17T04:32:20.055417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:15:33.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:18.098829+0000","last_change":"2024-05-17T04:31:39.894453+0000","last_active":"2024-05-17T04:33:18.098829+0000","last_peered":"2024-05-17T04:33:18.098829+0000","last_clean":"2024-05-17T04:33:18.098829+0000","last_became_active":"2024-05-17T04:31:39.893628+0000","last_became_peered":"2024-05-17T04:31:39.893628+0000","last_unstale":"2024-05-17T04:33:18.098829+0000","last_undegraded":"2024-05-17T04:33:18.098829+0000","last_fullsized":"2024-05-17T04:33:18.098829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:55:48.098182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059157+0000","last_change":"2024-05-17T04:31:42.939487+0000","last_active":"2024-05-17T04:32:20.059157+0000","last_peered":"2024-05-17T04:32:20.059157+0000","last_clean":"2024-05-17T04:32:20.059157+0000","last_became_active":"2024-05-17T04:31:42.938719+0000","last_became_peered":"2024-05-17T04:31:42.938719+0000","last_unstale":"2024-05-17T04:32:20.059157+0000","last_undegraded":"2024-05-17T04:32:20.059157+0000","last_fullsized":"2024-05-17T04:32:20.059157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:22:42.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056715+0000","last_change":"2024-05-17T04:31:55.998287+0000","last_active":"2024-05-17T04:32:20.056715+0000","last_peered":"2024-05-17T04:32:20.056715+0000","last_clean":"2024-05-17T04:32:20.056715+0000","last_became_active":"2024-05-17T04:31:55.997876+0000","last_became_peered":"2024-05-17T04:31:55.997876+0000","last_unstale":"2024-05-17T04:32:20.056715+0000","last_undegraded":"2024-05-17T04:32:20.056715+0000","last_fullsized":"2024-05-17T04:32:20.056715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:20.523375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.043308+0000","last_change":"2024-05-17T04:31:39.896995+0000","last_active":"2024-05-17T04:32:48.043308+0000","last_peered":"2024-05-17T04:32:48.043308+0000","last_clean":"2024-05-17T04:32:48.043308+0000","last_became_active":"2024-05-17T04:31:39.896607+0000","last_became_peered":"2024-05-17T04:31:39.896607+0000","last_unstale":"2024-05-17T04:32:48.043308+0000","last_undegraded":"2024-05-17T04:32:48.043308+0000","last_fullsized":"2024-05-17T04:32:48.043308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:13:26.761427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061923+0000","last_change":"2024-05-17T04:32:20.061507+0000","last_active":"2024-05-17T04:32:20.061923+0000","last_peered":"2024-05-17T04:32:20.061923+0000","last_clean":"2024-05-17T04:32:20.061923+0000","last_became_active":"2024-05-17T04:32:20.061245+0000","last_became_peered":"2024-05-17T04:32:20.061245+0000","last_unstale":"2024-05-17T04:32:20.061923+0000","last_undegraded":"2024-05-17T04:32:20.061923+0000","last_fullsized":"2024-05-17T04:32:20.061923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:43.800524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058971+0000","last_change":"2024-05-17T04:31:55.991340+0000","last_active":"2024-05-17T04:32:20.058971+0000","last_peered":"2024-05-17T04:32:20.058971+0000","last_clean":"2024-05-17T04:32:20.058971+0000","last_became_active":"2024-05-17T04:31:55.990733+0000","last_became_peered":"2024-05-17T04:31:55.990733+0000","last_unstale":"2024-05-17T04:32:20.058971+0000","last_undegraded":"2024-05-17T04:32:20.058971+0000","last_fullsized":"2024-05-17T04:32:20.058971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:01:47.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:50.037510+0000","last_change":"2024-05-17T04:32:20.070153+0000","last_active":"2024-05-17T04:32:50.037510+0000","last_peered":"2024-05-17T04:32:50.037510+0000","last_clean":"2024-05-17T04:32:50.037510+0000","last_became_active":"2024-05-17T04:32:20.062400+0000","last_became_peered":"2024-05-17T04:32:20.062400+0000","last_unstale":"2024-05-17T04:32:50.037510+0000","last_undegraded":"2024-05-17T04:32:50.037510+0000","last_fullsized":"2024-05-17T04:32:50.037510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:25:03.332079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.064121+0000","last_change":"2024-05-17T04:32:20.064121+0000","last_active":"2024-05-17T04:32:20.064121+0000","last_peered":"2024-05-17T04:32:20.064121+0000","last_clean":"2024-05-17T04:32:20.064121+0000","last_became_active":"2024-05-17T04:32:20.063707+0000","last_became_peered":"2024-05-17T04:32:20.063707+0000","last_unstale":"2024-05-17T04:32:20.064121+0000","last_undegraded":"2024-05-17T04:32:20.064121+0000","last_fullsized":"2024-05-17T04:32:20.064121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:57:45.362980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052409+0000","last_change":"2024-05-17T04:31:56.178977+0000","last_active":"2024-05-17T04:32:20.052409+0000","last_peered":"2024-05-17T04:32:20.052409+0000","last_clean":"2024-05-17T04:32:20.052409+0000","last_became_active":"2024-05-17T04:31:56.178452+0000","last_became_peered":"2024-05-17T04:31:56.178452+0000","last_unstale":"2024-05-17T04:32:20.052409+0000","last_undegraded":"2024-05-17T04:32:20.052409+0000","last_fullsized":"2024-05-17T04:32:20.052409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:14:19.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055838+0000","last_change":"2024-05-17T04:31:39.897118+0000","last_active":"2024-05-17T04:32:20.055838+0000","last_peered":"2024-05-17T04:32:20.055838+0000","last_clean":"2024-05-17T04:32:20.055838+0000","last_became_active":"2024-05-17T04:31:39.896761+0000","last_became_peered":"2024-05-17T04:31:39.896761+0000","last_unstale":"2024-05-17T04:32:20.055838+0000","last_undegraded":"2024-05-17T04:32:20.055838+0000","last_fullsized":"2024-05-17T04:32:20.055838+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:41:50.575131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055597+0000","last_change":"2024-05-17T04:31:42.930510+0000","last_active":"2024-05-17T04:32:20.055597+0000","last_peered":"2024-05-17T04:32:20.055597+0000","last_clean":"2024-05-17T04:32:20.055597+0000","last_became_active":"2024-05-17T04:31:42.930168+0000","last_became_peered":"2024-05-17T04:31:42.930168+0000","last_unstale":"2024-05-17T04:32:20.055597+0000","last_undegraded":"2024-05-17T04:32:20.055597+0000","last_fullsized":"2024-05-17T04:32:20.055597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:18:23.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056521+0000","last_change":"2024-05-17T04:31:55.996783+0000","last_active":"2024-05-17T04:32:20.056521+0000","last_peered":"2024-05-17T04:32:20.056521+0000","last_clean":"2024-05-17T04:32:20.056521+0000","last_became_active":"2024-05-17T04:31:55.996343+0000","last_became_peered":"2024-05-17T04:31:55.996343+0000","last_unstale":"2024-05-17T04:32:20.056521+0000","last_undegraded":"2024-05-17T04:32:20.056521+0000","last_fullsized":"2024-05-17T04:32:20.056521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:41:49.074496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060706+0000","last_change":"2024-05-17T04:31:39.890854+0000","last_active":"2024-05-17T04:32:20.060706+0000","last_peered":"2024-05-17T04:32:20.060706+0000","last_clean":"2024-05-17T04:32:20.060706+0000","last_became_active":"2024-05-17T04:31:39.890379+0000","last_became_peered":"2024-05-17T04:31:39.890379+0000","last_unstale":"2024-05-17T04:32:20.060706+0000","last_undegraded":"2024-05-17T04:32:20.060706+0000","last_fullsized":"2024-05-17T04:32:20.060706+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:06.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055604+0000","last_change":"2024-05-17T04:31:42.936992+0000","last_active":"2024-05-17T04:32:20.055604+0000","last_peered":"2024-05-17T04:32:20.055604+0000","last_clean":"2024-05-17T04:32:20.055604+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:32:20.055604+0000","last_undegraded":"2024-05-17T04:32:20.055604+0000","last_fullsized":"2024-05-17T04:32:20.055604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:27:33.067173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056460+0000","last_change":"2024-05-17T04:31:56.178921+0000","last_active":"2024-05-17T04:32:20.056460+0000","last_peered":"2024-05-17T04:32:20.056460+0000","last_clean":"2024-05-17T04:32:20.056460+0000","last_became_active":"2024-05-17T04:31:56.178430+0000","last_became_peered":"2024-05-17T04:31:56.178430+0000","last_unstale":"2024-05-17T04:32:20.056460+0000","last_undegraded":"2024-05-17T04:32:20.056460+0000","last_fullsized":"2024-05-17T04:32:20.056460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:56:37.980197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060976+0000","last_change":"2024-05-17T04:31:42.938586+0000","last_active":"2024-05-17T04:32:20.060976+0000","last_peered":"2024-05-17T04:32:20.060976+0000","last_clean":"2024-05-17T04:32:20.060976+0000","last_became_active":"2024-05-17T04:31:42.937861+0000","last_became_peered":"2024-05-17T04:31:42.937861+0000","last_unstale":"2024-05-17T04:32:20.060976+0000","last_undegraded":"2024-05-17T04:32:20.060976+0000","last_fullsized":"2024-05-17T04:32:20.060976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:03:11.790930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:14.092488+0000","last_change":"2024-05-17T04:31:39.896824+0000","last_active":"2024-05-17T04:33:14.092488+0000","last_peered":"2024-05-17T04:33:14.092488+0000","last_clean":"2024-05-17T04:33:14.092488+0000","last_became_active":"2024-05-17T04:31:39.896463+0000","last_became_peered":"2024-05-17T04:31:39.896463+0000","last_unstale":"2024-05-17T04:33:14.092488+0000","last_undegraded":"2024-05-17T04:33:14.092488+0000","last_fullsized":"2024-05-17T04:33:14.092488+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:36:08.351784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060596+0000","last_change":"2024-05-17T04:31:55.993336+0000","last_active":"2024-05-17T04:32:20.060596+0000","last_peered":"2024-05-17T04:32:20.060596+0000","last_clean":"2024-05-17T04:32:20.060596+0000","last_became_active":"2024-05-17T04:31:55.992851+0000","last_became_peered":"2024-05-17T04:31:55.992851+0000","last_unstale":"2024-05-17T04:32:20.060596+0000","last_undegraded":"2024-05-17T04:32:20.060596+0000","last_fullsized":"2024-05-17T04:32:20.060596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:07:47.563166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198117+0000","last_change":"2024-05-17T04:31:42.941862+0000","last_active":"2024-05-17T04:32:20.198117+0000","last_peered":"2024-05-17T04:32:20.198117+0000","last_clean":"2024-05-17T04:32:20.198117+0000","last_became_active":"2024-05-17T04:31:42.940304+0000","last_became_peered":"2024-05-17T04:31:42.940304+0000","last_unstale":"2024-05-17T04:32:20.198117+0000","last_undegraded":"2024-05-17T04:32:20.198117+0000","last_fullsized":"2024-05-17T04:32:20.198117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:46:14.569208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:50.052979+0000","last_change":"2024-05-17T04:31:39.894080+0000","last_active":"2024-05-17T04:32:50.052979+0000","last_peered":"2024-05-17T04:32:50.052979+0000","last_clean":"2024-05-17T04:32:50.052979+0000","last_became_active":"2024-05-17T04:31:39.893646+0000","last_became_peered":"2024-05-17T04:31:39.893646+0000","last_unstale":"2024-05-17T04:32:50.052979+0000","last_undegraded":"2024-05-17T04:32:50.052979+0000","last_fullsized":"2024-05-17T04:32:50.052979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:27:59.844497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058908+0000","last_change":"2024-05-17T04:30:11.017843+0000","last_active":"2024-05-17T04:32:20.058908+0000","last_peered":"2024-05-17T04:32:20.058908+0000","last_clean":"2024-05-17T04:32:20.058908+0000","last_became_active":"2024-05-17T04:30:11.013443+0000","last_became_peered":"2024-05-17T04:30:11.013443+0000","last_unstale":"2024-05-17T04:32:20.058908+0000","last_undegraded":"2024-05-17T04:32:20.058908+0000","last_fullsized":"2024-05-17T04:32:20.058908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_clean_scrub_stamp":"2024-05-17T04:26:43.066168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:27:18.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061659+0000","last_change":"2024-05-17T04:31:56.179071+0000","last_active":"2024-05-17T04:32:20.061659+0000","last_peered":"2024-05-17T04:32:20.061659+0000","last_clean":"2024-05-17T04:32:20.061659+0000","last_became_active":"2024-05-17T04:31:56.178500+0000","last_became_peered":"2024-05-17T04:31:56.178500+0000","last_unstale":"2024-05-17T04:32:20.061659+0000","last_undegraded":"2024-05-17T04:32:20.061659+0000","last_fullsized":"2024-05-17T04:32:20.061659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:48:02.144119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055528+0000","last_change":"2024-05-17T04:31:42.932232+0000","last_active":"2024-05-17T04:32:20.055528+0000","last_peered":"2024-05-17T04:32:20.055528+0000","last_clean":"2024-05-17T04:32:20.055528+0000","last_became_active":"2024-05-17T04:31:42.931915+0000","last_became_peered":"2024-05-17T04:31:42.931915+0000","last_unstale":"2024-05-17T04:32:20.055528+0000","last_undegraded":"2024-05-17T04:32:20.055528+0000","last_fullsized":"2024-05-17T04:32:20.055528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:06:57.693180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.041239+0000","last_change":"2024-05-17T04:31:39.890852+0000","last_active":"2024-05-17T04:32:48.041239+0000","last_peered":"2024-05-17T04:32:48.041239+0000","last_clean":"2024-05-17T04:32:48.041239+0000","last_became_active":"2024-05-17T04:31:39.890417+0000","last_became_peered":"2024-05-17T04:31:39.890417+0000","last_unstale":"2024-05-17T04:32:48.041239+0000","last_undegraded":"2024-05-17T04:32:48.041239+0000","last_fullsized":"2024-05-17T04:32:48.041239+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:45:47.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054730+0000","last_change":"2024-05-17T04:31:55.988931+0000","last_active":"2024-05-17T04:32:20.054730+0000","last_peered":"2024-05-17T04:32:20.054730+0000","last_clean":"2024-05-17T04:32:20.054730+0000","last_became_active":"2024-05-17T04:31:55.988366+0000","last_became_peered":"2024-05-17T04:31:55.988366+0000","last_unstale":"2024-05-17T04:32:20.054730+0000","last_undegraded":"2024-05-17T04:32:20.054730+0000","last_fullsized":"2024-05-17T04:32:20.054730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:36:30.684945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.036229+0000","last_change":"2024-05-17T04:32:20.059700+0000","last_active":"2024-05-17T04:32:47.036229+0000","last_peered":"2024-05-17T04:32:47.036229+0000","last_clean":"2024-05-17T04:32:47.036229+0000","last_became_active":"2024-05-17T04:32:20.059234+0000","last_became_peered":"2024-05-17T04:32:20.059234+0000","last_unstale":"2024-05-17T04:32:47.036229+0000","last_undegraded":"2024-05-17T04:32:47.036229+0000","last_fullsized":"2024-05-17T04:32:47.036229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:25:47.214832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056753+0000","last_change":"2024-05-17T04:31:42.926078+0000","last_active":"2024-05-17T04:32:20.056753+0000","last_peered":"2024-05-17T04:32:20.056753+0000","last_clean":"2024-05-17T04:32:20.056753+0000","last_became_active":"2024-05-17T04:31:42.925755+0000","last_became_peered":"2024-05-17T04:31:42.925755+0000","last_unstale":"2024-05-17T04:32:20.056753+0000","last_undegraded":"2024-05-17T04:32:20.056753+0000","last_fullsized":"2024-05-17T04:32:20.056753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:30:16.952810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.197720+0000","last_change":"2024-05-17T04:31:56.178973+0000","last_active":"2024-05-17T04:32:20.197720+0000","last_peered":"2024-05-17T04:32:20.197720+0000","last_clean":"2024-05-17T04:32:20.197720+0000","last_became_active":"2024-05-17T04:31:56.178359+0000","last_became_peered":"2024-05-17T04:31:56.178359+0000","last_unstale":"2024-05-17T04:32:20.197720+0000","last_undegraded":"2024-05-17T04:32:20.197720+0000","last_fullsized":"2024-05-17T04:32:20.197720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:22:48.921778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:33:12.943365+0000","last_change":"2024-05-17T04:31:39.887364+0000","last_active":"2024-05-17T04:33:12.943365+0000","last_peered":"2024-05-17T04:33:12.943365+0000","last_clean":"2024-05-17T04:33:12.943365+0000","last_became_active":"2024-05-17T04:31:39.886961+0000","last_became_peered":"2024-05-17T04:31:39.886961+0000","last_unstale":"2024-05-17T04:33:12.943365+0000","last_undegraded":"2024-05-17T04:33:12.943365+0000","last_fullsized":"2024-05-17T04:33:12.943365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:48:20.417903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198520+0000","last_change":"2024-05-17T04:31:42.933338+0000","last_active":"2024-05-17T04:32:20.198520+0000","last_peered":"2024-05-17T04:32:20.198520+0000","last_clean":"2024-05-17T04:32:20.198520+0000","last_became_active":"2024-05-17T04:31:42.931619+0000","last_became_peered":"2024-05-17T04:31:42.931619+0000","last_unstale":"2024-05-17T04:32:20.198520+0000","last_undegraded":"2024-05-17T04:32:20.198520+0000","last_fullsized":"2024-05-17T04:32:20.198520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:52.565132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056879+0000","last_change":"2024-05-17T04:31:55.990398+0000","last_active":"2024-05-17T04:32:20.056879+0000","last_peered":"2024-05-17T04:32:20.056879+0000","last_clean":"2024-05-17T04:32:20.056879+0000","last_became_active":"2024-05-17T04:31:55.989595+0000","last_became_peered":"2024-05-17T04:31:55.989595+0000","last_unstale":"2024-05-17T04:32:20.056879+0000","last_undegraded":"2024-05-17T04:32:20.056879+0000","last_fullsized":"2024-05-17T04:32:20.056879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:41:37.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.034597+0000","last_change":"2024-05-17T04:31:40.443503+0000","last_active":"2024-05-17T04:32:47.034597+0000","last_peered":"2024-05-17T04:32:47.034597+0000","last_clean":"2024-05-17T04:32:47.034597+0000","last_became_active":"2024-05-17T04:31:40.443094+0000","last_became_peered":"2024-05-17T04:31:40.443094+0000","last_unstale":"2024-05-17T04:32:47.034597+0000","last_undegraded":"2024-05-17T04:32:47.034597+0000","last_fullsized":"2024-05-17T04:32:47.034597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:03:26.837211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285380+0000","last_change":"2024-05-17T04:31:42.933715+0000","last_active":"2024-05-17T04:33:20.285380+0000","last_peered":"2024-05-17T04:33:20.285380+0000","last_clean":"2024-05-17T04:33:20.285380+0000","last_became_active":"2024-05-17T04:31:42.933360+0000","last_became_peered":"2024-05-17T04:31:42.933360+0000","last_unstale":"2024-05-17T04:33:20.285380+0000","last_undegraded":"2024-05-17T04:33:20.285380+0000","last_fullsized":"2024-05-17T04:33:20.285380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:02:30.568910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.058554+0000","last_change":"2024-05-17T04:31:56.179224+0000","last_active":"2024-05-17T04:32:20.058554+0000","last_peered":"2024-05-17T04:32:20.058554+0000","last_clean":"2024-05-17T04:32:20.058554+0000","last_became_active":"2024-05-17T04:31:56.178458+0000","last_became_peered":"2024-05-17T04:31:56.178458+0000","last_unstale":"2024-05-17T04:32:20.058554+0000","last_undegraded":"2024-05-17T04:32:20.058554+0000","last_fullsized":"2024-05-17T04:32:20.058554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:34:51.303044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037558+0000","last_change":"2024-05-17T04:31:39.896901+0000","last_active":"2024-05-17T04:32:47.037558+0000","last_peered":"2024-05-17T04:32:47.037558+0000","last_clean":"2024-05-17T04:32:47.037558+0000","last_became_active":"2024-05-17T04:31:39.896580+0000","last_became_peered":"2024-05-17T04:31:39.896580+0000","last_unstale":"2024-05-17T04:32:47.037558+0000","last_undegraded":"2024-05-17T04:32:47.037558+0000","last_fullsized":"2024-05-17T04:32:47.037558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:26:24.317276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.052350+0000","last_change":"2024-05-17T04:31:42.935437+0000","last_active":"2024-05-17T04:32:20.052350+0000","last_peered":"2024-05-17T04:32:20.052350+0000","last_clean":"2024-05-17T04:32:20.052350+0000","last_became_active":"2024-05-17T04:31:42.935067+0000","last_became_peered":"2024-05-17T04:31:42.935067+0000","last_unstale":"2024-05-17T04:32:20.052350+0000","last_undegraded":"2024-05-17T04:32:20.052350+0000","last_fullsized":"2024-05-17T04:32:20.052350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:56:20.766093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055636+0000","last_change":"2024-05-17T04:31:55.989080+0000","last_active":"2024-05-17T04:32:20.055636+0000","last_peered":"2024-05-17T04:32:20.055636+0000","last_clean":"2024-05-17T04:32:20.055636+0000","last_became_active":"2024-05-17T04:31:55.988657+0000","last_became_peered":"2024-05-17T04:31:55.988657+0000","last_unstale":"2024-05-17T04:32:20.055636+0000","last_undegraded":"2024-05-17T04:32:20.055636+0000","last_fullsized":"2024-05-17T04:32:20.055636+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:42:06.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.040252+0000","last_change":"2024-05-17T04:31:39.893877+0000","last_active":"2024-05-17T04:32:47.040252+0000","last_peered":"2024-05-17T04:32:47.040252+0000","last_clean":"2024-05-17T04:32:47.040252+0000","last_became_active":"2024-05-17T04:31:39.893479+0000","last_became_peered":"2024-05-17T04:31:39.893479+0000","last_unstale":"2024-05-17T04:32:47.040252+0000","last_undegraded":"2024-05-17T04:32:47.040252+0000","last_fullsized":"2024-05-17T04:32:47.040252+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:46:21.163132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054661+0000","last_change":"2024-05-17T04:31:42.934449+0000","last_active":"2024-05-17T04:32:20.054661+0000","last_peered":"2024-05-17T04:32:20.054661+0000","last_clean":"2024-05-17T04:32:20.054661+0000","last_became_active":"2024-05-17T04:31:42.934054+0000","last_became_peered":"2024-05-17T04:31:42.934054+0000","last_unstale":"2024-05-17T04:32:20.054661+0000","last_undegraded":"2024-05-17T04:32:20.054661+0000","last_fullsized":"2024-05-17T04:32:20.054661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:35.344333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059309+0000","last_change":"2024-05-17T04:31:55.986935+0000","last_active":"2024-05-17T04:32:20.059309+0000","last_peered":"2024-05-17T04:32:20.059309+0000","last_clean":"2024-05-17T04:32:20.059309+0000","last_became_active":"2024-05-17T04:31:55.986471+0000","last_became_peered":"2024-05-17T04:31:55.986471+0000","last_unstale":"2024-05-17T04:32:20.059309+0000","last_undegraded":"2024-05-17T04:32:20.059309+0000","last_fullsized":"2024-05-17T04:32:20.059309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:04:09.479135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057278+0000","last_change":"2024-05-17T04:31:39.890000+0000","last_active":"2024-05-17T04:32:20.057278+0000","last_peered":"2024-05-17T04:32:20.057278+0000","last_clean":"2024-05-17T04:32:20.057278+0000","last_became_active":"2024-05-17T04:31:39.887979+0000","last_became_peered":"2024-05-17T04:31:39.887979+0000","last_unstale":"2024-05-17T04:32:20.057278+0000","last_undegraded":"2024-05-17T04:32:20.057278+0000","last_fullsized":"2024-05-17T04:32:20.057278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:44:13.580387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198348+0000","last_change":"2024-05-17T04:31:42.943980+0000","last_active":"2024-05-17T04:32:20.198348+0000","last_peered":"2024-05-17T04:32:20.198348+0000","last_clean":"2024-05-17T04:32:20.198348+0000","last_became_active":"2024-05-17T04:31:42.943647+0000","last_became_peered":"2024-05-17T04:31:42.943647+0000","last_unstale":"2024-05-17T04:32:20.198348+0000","last_undegraded":"2024-05-17T04:32:20.198348+0000","last_fullsized":"2024-05-17T04:32:20.198348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:17:15.909198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060156+0000","last_change":"2024-05-17T04:31:56.178896+0000","last_active":"2024-05-17T04:32:20.060156+0000","last_peered":"2024-05-17T04:32:20.060156+0000","last_clean":"2024-05-17T04:32:20.060156+0000","last_became_active":"2024-05-17T04:31:56.178298+0000","last_became_peered":"2024-05-17T04:31:56.178298+0000","last_unstale":"2024-05-17T04:32:20.060156+0000","last_undegraded":"2024-05-17T04:32:20.060156+0000","last_fullsized":"2024-05-17T04:32:20.060156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:30:32.759024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:47.037577+0000","last_change":"2024-05-17T04:31:39.894705+0000","last_active":"2024-05-17T04:32:47.037577+0000","last_peered":"2024-05-17T04:32:47.037577+0000","last_clean":"2024-05-17T04:32:47.037577+0000","last_became_active":"2024-05-17T04:31:39.894312+0000","last_became_peered":"2024-05-17T04:31:39.894312+0000","last_unstale":"2024-05-17T04:32:47.037577+0000","last_undegraded":"2024-05-17T04:32:47.037577+0000","last_fullsized":"2024-05-17T04:32:47.037577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:41:13.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060452+0000","last_change":"2024-05-17T04:31:42.941171+0000","last_active":"2024-05-17T04:32:20.060452+0000","last_peered":"2024-05-17T04:32:20.060452+0000","last_clean":"2024-05-17T04:32:20.060452+0000","last_became_active":"2024-05-17T04:31:42.940724+0000","last_became_peered":"2024-05-17T04:31:42.940724+0000","last_unstale":"2024-05-17T04:32:20.060452+0000","last_undegraded":"2024-05-17T04:32:20.060452+0000","last_fullsized":"2024-05-17T04:32:20.060452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:26:08.933000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054328+0000","last_change":"2024-05-17T04:31:55.992427+0000","last_active":"2024-05-17T04:32:20.054328+0000","last_peered":"2024-05-17T04:32:20.054328+0000","last_clean":"2024-05-17T04:32:20.054328+0000","last_became_active":"2024-05-17T04:31:55.991936+0000","last_became_peered":"2024-05-17T04:31:55.991936+0000","last_unstale":"2024-05-17T04:32:20.054328+0000","last_undegraded":"2024-05-17T04:32:20.054328+0000","last_fullsized":"2024-05-17T04:32:20.054328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:30:01.610347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:48.042774+0000","last_change":"2024-05-17T04:32:20.062349+0000","last_active":"2024-05-17T04:32:48.042774+0000","last_peered":"2024-05-17T04:32:48.042774+0000","last_clean":"2024-05-17T04:32:48.042774+0000","last_became_active":"2024-05-17T04:32:20.061708+0000","last_became_peered":"2024-05-17T04:32:20.061708+0000","last_unstale":"2024-05-17T04:32:48.042774+0000","last_undegraded":"2024-05-17T04:32:48.042774+0000","last_fullsized":"2024-05-17T04:32:48.042774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:51:02.946670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060650+0000","last_change":"2024-05-17T04:31:42.931998+0000","last_active":"2024-05-17T04:32:20.060650+0000","last_peered":"2024-05-17T04:32:20.060650+0000","last_clean":"2024-05-17T04:32:20.060650+0000","last_became_active":"2024-05-17T04:31:42.929898+0000","last_became_peered":"2024-05-17T04:31:42.929898+0000","last_unstale":"2024-05-17T04:32:20.060650+0000","last_undegraded":"2024-05-17T04:32:20.060650+0000","last_fullsized":"2024-05-17T04:32:20.060650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:55:35.349637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285002+0000","last_change":"2024-05-17T04:31:55.990093+0000","last_active":"2024-05-17T04:33:20.285002+0000","last_peered":"2024-05-17T04:33:20.285002+0000","last_clean":"2024-05-17T04:33:20.285002+0000","last_became_active":"2024-05-17T04:31:55.989665+0000","last_became_peered":"2024-05-17T04:31:55.989665+0000","last_unstale":"2024-05-17T04:33:20.285002+0000","last_undegraded":"2024-05-17T04:33:20.285002+0000","last_fullsized":"2024-05-17T04:33:20.285002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:37:43.727322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060975+0000","last_change":"2024-05-17T04:31:39.893601+0000","last_active":"2024-05-17T04:32:20.060975+0000","last_peered":"2024-05-17T04:32:20.060975+0000","last_clean":"2024-05-17T04:32:20.060975+0000","last_became_active":"2024-05-17T04:31:39.893141+0000","last_became_peered":"2024-05-17T04:31:39.893141+0000","last_unstale":"2024-05-17T04:32:20.060975+0000","last_undegraded":"2024-05-17T04:32:20.060975+0000","last_fullsized":"2024-05-17T04:32:20.060975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:05:09.173607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060155+0000","last_change":"2024-05-17T04:31:42.932083+0000","last_active":"2024-05-17T04:32:20.060155+0000","last_peered":"2024-05-17T04:32:20.060155+0000","last_clean":"2024-05-17T04:32:20.060155+0000","last_became_active":"2024-05-17T04:31:42.931520+0000","last_became_peered":"2024-05-17T04:31:42.931520+0000","last_unstale":"2024-05-17T04:32:20.060155+0000","last_undegraded":"2024-05-17T04:32:20.060155+0000","last_fullsized":"2024-05-17T04:32:20.060155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:47:55.057982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.059905+0000","last_change":"2024-05-17T04:31:55.990315+0000","last_active":"2024-05-17T04:32:20.059905+0000","last_peered":"2024-05-17T04:32:20.059905+0000","last_clean":"2024-05-17T04:32:20.059905+0000","last_became_active":"2024-05-17T04:31:55.989565+0000","last_became_peered":"2024-05-17T04:31:55.989565+0000","last_unstale":"2024-05-17T04:32:20.059905+0000","last_undegraded":"2024-05-17T04:32:20.059905+0000","last_fullsized":"2024-05-17T04:32:20.059905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:03.699211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056710+0000","last_change":"2024-05-17T04:31:39.890621+0000","last_active":"2024-05-17T04:32:20.056710+0000","last_peered":"2024-05-17T04:32:20.056710+0000","last_clean":"2024-05-17T04:32:20.056710+0000","last_became_active":"2024-05-17T04:31:39.890180+0000","last_became_peered":"2024-05-17T04:31:39.890180+0000","last_unstale":"2024-05-17T04:32:20.056710+0000","last_undegraded":"2024-05-17T04:32:20.056710+0000","last_fullsized":"2024-05-17T04:32:20.056710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:42.018482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.060778+0000","last_change":"2024-05-17T04:31:42.928951+0000","last_active":"2024-05-17T04:32:20.060778+0000","last_peered":"2024-05-17T04:32:20.060778+0000","last_clean":"2024-05-17T04:32:20.060778+0000","last_became_active":"2024-05-17T04:31:42.928382+0000","last_became_peered":"2024-05-17T04:31:42.928382+0000","last_unstale":"2024-05-17T04:32:20.060778+0000","last_undegraded":"2024-05-17T04:32:20.060778+0000","last_fullsized":"2024-05-17T04:32:20.060778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:36.307351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054747+0000","last_change":"2024-05-17T04:31:42.926377+0000","last_active":"2024-05-17T04:32:20.054747+0000","last_peered":"2024-05-17T04:32:20.054747+0000","last_clean":"2024-05-17T04:32:20.054747+0000","last_became_active":"2024-05-17T04:31:42.926007+0000","last_became_peered":"2024-05-17T04:31:42.926007+0000","last_unstale":"2024-05-17T04:32:20.054747+0000","last_undegraded":"2024-05-17T04:32:20.054747+0000","last_fullsized":"2024-05-17T04:32:20.054747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:43:45.456144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057772+0000","last_change":"2024-05-17T04:31:42.932742+0000","last_active":"2024-05-17T04:32:20.057772+0000","last_peered":"2024-05-17T04:32:20.057772+0000","last_clean":"2024-05-17T04:32:20.057772+0000","last_became_active":"2024-05-17T04:31:42.932374+0000","last_became_peered":"2024-05-17T04:31:42.932374+0000","last_unstale":"2024-05-17T04:32:20.057772+0000","last_undegraded":"2024-05-17T04:32:20.057772+0000","last_fullsized":"2024-05-17T04:32:20.057772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:37.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057008+0000","last_change":"2024-05-17T04:31:42.940870+0000","last_active":"2024-05-17T04:32:20.057008+0000","last_peered":"2024-05-17T04:32:20.057008+0000","last_clean":"2024-05-17T04:32:20.057008+0000","last_became_active":"2024-05-17T04:31:42.940463+0000","last_became_peered":"2024-05-17T04:31:42.940463+0000","last_unstale":"2024-05-17T04:32:20.057008+0000","last_undegraded":"2024-05-17T04:32:20.057008+0000","last_fullsized":"2024-05-17T04:32:20.057008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:48:52.684601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.285185+0000","last_change":"2024-05-17T04:31:42.936055+0000","last_active":"2024-05-17T04:33:20.285185+0000","last_peered":"2024-05-17T04:33:20.285185+0000","last_clean":"2024-05-17T04:33:20.285185+0000","last_became_active":"2024-05-17T04:31:42.934013+0000","last_became_peered":"2024-05-17T04:31:42.934013+0000","last_unstale":"2024-05-17T04:33:20.285185+0000","last_undegraded":"2024-05-17T04:33:20.285185+0000","last_fullsized":"2024-05-17T04:33:20.285185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:03:10.502727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056778+0000","last_change":"2024-05-17T04:31:42.942930+0000","last_active":"2024-05-17T04:32:20.056778+0000","last_peered":"2024-05-17T04:32:20.056778+0000","last_clean":"2024-05-17T04:32:20.056778+0000","last_became_active":"2024-05-17T04:31:42.942585+0000","last_became_peered":"2024-05-17T04:31:42.942585+0000","last_unstale":"2024-05-17T04:32:20.056778+0000","last_undegraded":"2024-05-17T04:32:20.056778+0000","last_fullsized":"2024-05-17T04:32:20.056778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:28:45.780505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056274+0000","last_change":"2024-05-17T04:31:42.923752+0000","last_active":"2024-05-17T04:32:20.056274+0000","last_peered":"2024-05-17T04:32:20.056274+0000","last_clean":"2024-05-17T04:32:20.056274+0000","last_became_active":"2024-05-17T04:31:42.923375+0000","last_became_peered":"2024-05-17T04:31:42.923375+0000","last_unstale":"2024-05-17T04:32:20.056274+0000","last_undegraded":"2024-05-17T04:32:20.056274+0000","last_fullsized":"2024-05-17T04:32:20.056274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:32:13.755314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.057926+0000","last_change":"2024-05-17T04:31:42.928666+0000","last_active":"2024-05-17T04:32:20.057926+0000","last_peered":"2024-05-17T04:32:20.057926+0000","last_clean":"2024-05-17T04:32:20.057926+0000","last_became_active":"2024-05-17T04:31:42.928291+0000","last_became_peered":"2024-05-17T04:31:42.928291+0000","last_unstale":"2024-05-17T04:32:20.057926+0000","last_undegraded":"2024-05-17T04:32:20.057926+0000","last_fullsized":"2024-05-17T04:32:20.057926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:12:32.253374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198633+0000","last_change":"2024-05-17T04:31:42.926316+0000","last_active":"2024-05-17T04:32:20.198633+0000","last_peered":"2024-05-17T04:32:20.198633+0000","last_clean":"2024-05-17T04:32:20.198633+0000","last_became_active":"2024-05-17T04:31:42.925823+0000","last_became_peered":"2024-05-17T04:31:42.925823+0000","last_unstale":"2024-05-17T04:32:20.198633+0000","last_undegraded":"2024-05-17T04:32:20.198633+0000","last_fullsized":"2024-05-17T04:32:20.198633+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:16:15.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.061004+0000","last_change":"2024-05-17T04:31:42.945587+0000","last_active":"2024-05-17T04:32:20.061004+0000","last_peered":"2024-05-17T04:32:20.061004+0000","last_clean":"2024-05-17T04:32:20.061004+0000","last_became_active":"2024-05-17T04:31:42.945238+0000","last_became_peered":"2024-05-17T04:31:42.945238+0000","last_unstale":"2024-05-17T04:32:20.061004+0000","last_undegraded":"2024-05-17T04:32:20.061004+0000","last_fullsized":"2024-05-17T04:32:20.061004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:17:29.584023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.198069+0000","last_change":"2024-05-17T04:31:42.925883+0000","last_active":"2024-05-17T04:32:20.198069+0000","last_peered":"2024-05-17T04:32:20.198069+0000","last_clean":"2024-05-17T04:32:20.198069+0000","last_became_active":"2024-05-17T04:31:42.925506+0000","last_became_peered":"2024-05-17T04:31:42.925506+0000","last_unstale":"2024-05-17T04:32:20.198069+0000","last_undegraded":"2024-05-17T04:32:20.198069+0000","last_fullsized":"2024-05-17T04:32:20.198069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:31:16.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.055198+0000","last_change":"2024-05-17T04:31:42.928444+0000","last_active":"2024-05-17T04:32:20.055198+0000","last_peered":"2024-05-17T04:32:20.055198+0000","last_clean":"2024-05-17T04:32:20.055198+0000","last_became_active":"2024-05-17T04:31:42.928060+0000","last_became_peered":"2024-05-17T04:31:42.928060+0000","last_unstale":"2024-05-17T04:32:20.055198+0000","last_undegraded":"2024-05-17T04:32:20.055198+0000","last_fullsized":"2024-05-17T04:32:20.055198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:47:29.714401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.056251+0000","last_change":"2024-05-17T04:31:42.937010+0000","last_active":"2024-05-17T04:32:20.056251+0000","last_peered":"2024-05-17T04:32:20.056251+0000","last_clean":"2024-05-17T04:32:20.056251+0000","last_became_active":"2024-05-17T04:31:42.936675+0000","last_became_peered":"2024-05-17T04:31:42.936675+0000","last_unstale":"2024-05-17T04:32:20.056251+0000","last_undegraded":"2024-05-17T04:32:20.056251+0000","last_fullsized":"2024-05-17T04:32:20.056251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:06:22.421966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-17T04:32:20.054911+0000","last_change":"2024-05-17T04:31:42.924943+0000","last_active":"2024-05-17T04:32:20.054911+0000","last_peered":"2024-05-17T04:32:20.054911+0000","last_clean":"2024-05-17T04:32:20.054911+0000","last_became_active":"2024-05-17T04:31:42.924578+0000","last_became_peered":"2024-05-17T04:31:42.924578+0000","last_unstale":"2024-05-17T04:32:20.054911+0000","last_undegraded":"2024-05-17T04:32:20.054911+0000","last_fullsized":"2024-05-17T04:32:20.054911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:22:58.614038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284318,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":204,"num_read_kb":171,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":195127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":496276,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1145963,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070510,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":500564,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":496344,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462968,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1092688,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":491644,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":554208,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":493032,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":550520,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":491838,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":496685,"data_compressed":26383,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":1087885,"data_compressed":62531,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T04:33:23.224 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T04:33:23.224 INFO:tasks.cephadm:Setting up client nodes... 2024-05-17T04:33:23.225 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-17T04:33:23.481 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:24.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:23 smithi050 ceph-mon[31739]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:24.309 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:23 smithi050 ceph-mon[31739]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 4 op/s 2024-05-17T04:33:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:23 smithi105 ceph-mon[34795]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:23 smithi105 ceph-mon[34795]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 4 op/s 2024-05-17T04:33:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:23 smithi110 ceph-mon[35497]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:33:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:23 smithi110 ceph-mon[35497]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 4 op/s 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 -- 172.21.15.50:0/3827940345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40102b90 msgr2=0x7fac40102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 --2- 172.21.15.50:0/3827940345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40102b90 0x7fac40102fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac300099c0 tx=0x7fac30009cd0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 -- 172.21.15.50:0/3827940345 shutdown_connections 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 --2- 172.21.15.50:0/3827940345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 0x7fac4010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 --2- 172.21.15.50:0/3827940345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40102b90 0x7fac40102fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 --2- 172.21.15.50:0/3827940345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40103cc0 0x7fac401040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 -- 172.21.15.50:0/3827940345 >> 172.21.15.50:0/3827940345 conn(0x7fac400fe3c0 msgr2=0x7fac40100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 -- 172.21.15.50:0/3827940345 shutdown_connections 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.371+0000 7fac45350700 1 -- 172.21.15.50:0/3827940345 wait complete. 2024-05-17T04:33:25.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.372+0000 7fac45350700 1 Processor -- start 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.372+0000 7fac45350700 1 -- start start 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.372+0000 7fac45350700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40102b90 0x7fac40199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.372+0000 7fac45350700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.372+0000 7fac45350700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 0x7fac401a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac45350700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010e490 con 0x7fac40103cc0 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac45350700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010e310 con 0x7fac40102b90 2024-05-17T04:33:25.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac45350700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac4010e610 con 0x7fac40104660 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50042/0 (socket says 172.21.15.50:50042) 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 -- 172.21.15.50:0/3612602373 learned_addr learned my addr 172.21.15.50:0/3612602373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3f7fe700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 0x7fac401a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3effd700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40102b90 0x7fac40199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 -- 172.21.15.50:0/3612602373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 msgr2=0x7fac401a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 0x7fac401a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 -- 172.21.15.50:0/3612602373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40102b90 msgr2=0x7fac40199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40102b90 0x7fac40199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 -- 172.21.15.50:0/3612602373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac401a0bd0 con 0x7fac40103cc0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.373+0000 7fac3e7fc700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac34009970 tx=0x7fac34009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.374+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3401c070 con 0x7fac40103cc0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.374+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac3400d260 con 0x7fac40103cc0 2024-05-17T04:33:25.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.374+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34017930 con 0x7fac40103cc0 2024-05-17T04:33:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.374+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4006c0a0 con 0x7fac40103cc0 2024-05-17T04:33:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.374+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4006c590 con 0x7fac40103cc0 2024-05-17T04:33:25.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.375+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac40102fa0 con 0x7fac40103cc0 2024-05-17T04:33:25.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.377+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fac3401f070 con 0x7fac40103cc0 2024-05-17T04:33:25.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.378+0000 7fac27fff700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 0x7fac280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:25.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.378+0000 7fac3effd700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 0x7fac280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:25.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.378+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fac340e38d0 con 0x7fac40103cc0 2024-05-17T04:33:25.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.378+0000 7fac3effd700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 0x7fac280b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac30009990 tx=0x7fac30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:25.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.381+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac340ab0a0 con 0x7fac40103cc0 2024-05-17T04:33:25.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.681+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fac40109cc0 con 0x7fac40103cc0 2024-05-17T04:33:25.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.684+0000 7fac27fff700 1 -- 172.21.15.50:0/3612602373 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fac340aa9a0 con 0x7fac40103cc0 2024-05-17T04:33:25.685 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2024-05-17T04:33:25.685 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAV3kZm/Im3KBAAbyLOXYWgAHUzEALVG9yPOg== 2024-05-17T04:33:25.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 msgr2=0x7fac280b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:25.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 0x7fac280b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac30009990 tx=0x7fac30005fb0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 msgr2=0x7fac401995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fac34009970 tx=0x7fac34009c80 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 shutdown_connections 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac40104660 0x7fac401a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac40102b90 0x7fac40199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fac280b2990 0x7fac280b4ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 --2- 172.21.15.50:0/3612602373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac40103cc0 0x7fac401995b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 >> 172.21.15.50:0/3612602373 conn(0x7fac400fe3c0 msgr2=0x7fac40106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 shutdown_connections 2024-05-17T04:33:25.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:25.687+0000 7fac45350700 1 -- 172.21.15.50:0/3612602373 wait complete. 2024-05-17T04:33:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:25 smithi105 ceph-mon[34795]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:26.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:25 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T04:33:26.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:25 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T04:33:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:25 smithi110 ceph-mon[35497]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:25 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T04:33:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:25 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T04:33:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:25 smithi050 ceph-mon[31739]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:25 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-17T04:33:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:25 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3612602373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-17T04:33:26.406 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:33:26.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-17T04:33:26.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-17T04:33:26.450 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-17T04:33:26.451 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-17T04:33:26.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-17T04:33:26.743 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:28.282 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:27 smithi050 ceph-mon[31739]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:27 smithi105 ceph-mon[34795]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:27 smithi110 ceph-mon[35497]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 -- 172.21.15.50:0/978907136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4108830 msgr2=0x7fb0d4108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 --2- 172.21.15.50:0/978907136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4108830 0x7fb0d4108c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009990 tx=0x7fb0c4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 -- 172.21.15.50:0/978907136 shutdown_connections 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 --2- 172.21.15.50:0/978907136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4109180 0x7fb0d4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 --2- 172.21.15.50:0/978907136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4103f00 0x7fb0d4104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 --2- 172.21.15.50:0/978907136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4108830 0x7fb0d4108c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.528+0000 7fb0db4db700 1 -- 172.21.15.50:0/978907136 >> 172.21.15.50:0/978907136 conn(0x7fb0d40ff9c0 msgr2=0x7fb0d4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.529+0000 7fb0db4db700 1 -- 172.21.15.50:0/978907136 shutdown_connections 2024-05-17T04:33:28.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.529+0000 7fb0db4db700 1 -- 172.21.15.50:0/978907136 wait complete. 2024-05-17T04:33:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.529+0000 7fb0db4db700 1 Processor -- start 2024-05-17T04:33:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.529+0000 7fb0db4db700 1 -- start start 2024-05-17T04:33:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4108830 0x7fb0d41a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:28.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4109180 0x7fb0d419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4113c40 con 0x7fb0d4103f00 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4113ac0 con 0x7fb0d4108830 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0db4db700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4113dc0 con 0x7fb0d4109180 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50058/0 (socket says 172.21.15.50:50058) 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 -- 172.21.15.50:0/826967116 learned_addr learned my addr 172.21.15.50:0/826967116 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9a78700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4109180 0x7fb0d419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d8a76700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4108830 0x7fb0d41a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 -- 172.21.15.50:0/826967116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4109180 msgr2=0x7fb0d419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4109180 0x7fb0d419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 -- 172.21.15.50:0/826967116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4108830 msgr2=0x7fb0d41a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4108830 0x7fb0d41a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 -- 172.21.15.50:0/826967116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d419cdb0 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.530+0000 7fb0d9277700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d950 tx=0x7fb0d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.531+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0016070 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.531+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0d0009dd0 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.531+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d419d0a0 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.531+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000f3c0 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.531+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a9330 con 0x7fb0d4103f00 2024-05-17T04:33:28.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.532+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d41053b0 con 0x7fb0d4103f00 2024-05-17T04:33:28.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.535+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb0d0019070 con 0x7fb0d4103f00 2024-05-17T04:33:28.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.536+0000 7fb0ca7fc700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 0x7fb0c00b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:28.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.536+0000 7fb0d8a76700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 0x7fb0c00b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:28.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.536+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb0d00e2260 con 0x7fb0d4103f00 2024-05-17T04:33:28.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.536+0000 7fb0d8a76700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 0x7fb0c00b4f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0d419deb0 tx=0x7fb0c400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:28.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.538+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0d00a9b10 con 0x7fb0d4103f00 2024-05-17T04:33:28.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.817+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0d41a9920 con 0x7fb0d4103f00 2024-05-17T04:33:28.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.822+0000 7fb0ca7fc700 1 -- 172.21.15.50:0/826967116 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7fb0d00a9410 con 0x7fb0d4103f00 2024-05-17T04:33:28.826 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 msgr2=0x7fb0c00b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 0x7fb0c00b4f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0d419deb0 tx=0x7fb0c400b560 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 msgr2=0x7fb0d41a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d950 tx=0x7fb0d000dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 shutdown_connections 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0d4109180 0x7fb0d419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb0c00b2a60 0x7fb0c00b4f70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0d4103f00 0x7fb0d41a2580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 --2- 172.21.15.50:0/826967116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d4108830 0x7fb0d41a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.825+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 >> 172.21.15.50:0/826967116 conn(0x7fb0d40ff9c0 msgr2=0x7fb0d410a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.826+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 shutdown_connections 2024-05-17T04:33:28.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:28.826+0000 7fb0db4db700 1 -- 172.21.15.50:0/826967116 wait complete. 2024-05-17T04:33:29.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:28 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/826967116' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T04:33:29.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:28 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/826967116' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T04:33:29.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:28 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/826967116' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-17T04:33:29.545 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":1662041432},{"type":"v1","addr":"172.21.15.50:6801","nonce":1662041432}]},"active_addr":"172.21.15.50:6801/1662041432","active_change":"2024-05-17T04:24:18.593502+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.50:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1804966223}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":869091459}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1594728217}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":1063028425}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3494339650}]}]} 2024-05-17T04:33:29.550 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-17T04:33:29.550 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-17T04:33:29.551 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:33:29.805 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:30.193 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:29 smithi050 ceph-mon[31739]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-17T04:33:30.218 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:29 smithi110 ceph-mon[35497]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-17T04:33:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:29 smithi105 ceph-mon[34795]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3 op/s 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 -- 172.21.15.50:0/3324078867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 msgr2=0x7f9540107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 --2- 172.21.15.50:0/3324078867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f9540107b40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9534009990 tx=0x7f9534009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 -- 172.21.15.50:0/3324078867 shutdown_connections 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 --2- 172.21.15.50:0/3324078867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540108080 0x7f954010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 --2- 172.21.15.50:0/3324078867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540102e00 0x7f9540103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 --2- 172.21.15.50:0/3324078867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f9540107b40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 -- 172.21.15.50:0/3324078867 >> 172.21.15.50:0/3324078867 conn(0x7f9540079bd0 msgr2=0x7f9540079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 -- 172.21.15.50:0/3324078867 shutdown_connections 2024-05-17T04:33:31.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.577+0000 7f95464d0700 1 -- 172.21.15.50:0/3324078867 wait complete. 2024-05-17T04:33:31.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 Processor -- start 2024-05-17T04:33:31.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 -- start start 2024-05-17T04:33:31.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540102e00 0x7f954019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:31.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:31.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540108080 0x7f954019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:31.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9540112b40 con 0x7f9540107730 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95401129c0 con 0x7f9540102e00 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f95464d0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9540112cc0 con 0x7f9540108080 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f953f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f953f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48868/0 (socket says 172.21.15.50:48868) 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.578+0000 7f953f7fe700 1 -- 172.21.15.50:0/25767119 learned_addr learned my addr 172.21.15.50:0/25767119 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953ffff700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540102e00 0x7f954019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f9544a6d700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540108080 0x7f954019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 -- 172.21.15.50:0/25767119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540108080 msgr2=0x7f954019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540108080 0x7f954019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 -- 172.21.15.50:0/25767119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540102e00 msgr2=0x7f954019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:31.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540102e00 0x7f954019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 -- 172.21.15.50:0/25767119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a6920 con 0x7f9540107730 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953f7fe700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95340059d0 tx=0x7f953400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534027070 con 0x7f9540107730 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9534022470 con 0x7f9540107730 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a6bb0 con 0x7f9540107730 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534017650 con 0x7f9540107730 2024-05-17T04:33:31.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.579+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a7000 con 0x7f9540107730 2024-05-17T04:33:31.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.581+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540103210 con 0x7f9540107730 2024-05-17T04:33:31.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.582+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9534009d10 con 0x7f9540107730 2024-05-17T04:33:31.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.583+0000 7f953d7fa700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 0x7f95280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:31.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.583+0000 7f953ffff700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 0x7f95280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:31.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.583+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f95340e3290 con 0x7f9540107730 2024-05-17T04:33:31.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.583+0000 7f953ffff700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 0x7f95280b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9530005950 tx=0x7f953000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:31.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.587+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9534013070 con 0x7f9540107730 2024-05-17T04:33:31.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.824+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f954006bc70 con 0x7f9540107730 2024-05-17T04:33:31.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.826+0000 7f953d7fa700 1 -- 172.21.15.50:0/25767119 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7f95340aab50 con 0x7f9540107730 2024-05-17T04:33:31.827 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:31.827 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 msgr2=0x7f95280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 0x7f95280b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9530005950 tx=0x7f953000b410 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 msgr2=0x7f954019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95340059d0 tx=0x7f953400bfc0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 shutdown_connections 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9540108080 0x7f954019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9540102e00 0x7f954019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f95280b2910 0x7f95280b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 --2- 172.21.15.50:0/25767119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9540107730 0x7f954019bb10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 >> 172.21.15.50:0/25767119 conn(0x7f9540079bd0 msgr2=0x7f9540105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 shutdown_connections 2024-05-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:31.828+0000 7f95464d0700 1 -- 172.21.15.50:0/25767119 wait complete. 2024-05-17T04:33:32.199 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:31 smithi050 ceph-mon[31739]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:32.199 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:31 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/25767119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:31 smithi105 ceph-mon[34795]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:31 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/25767119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:31 smithi110 ceph-mon[35497]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:31 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/25767119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:32.565 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-17T04:33:32.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:33:32.823 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:34.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:33 smithi105 ceph-mon[34795]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:33 smithi110 ceph-mon[35497]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:33 smithi050 ceph-mon[31739]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:34.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 -- 172.21.15.50:0/3153116302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 msgr2=0x7f22bc105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 --2- 172.21.15.50:0/3153116302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc105230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009990 tx=0x7f22b8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 -- 172.21.15.50:0/3153116302 shutdown_connections 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 --2- 172.21.15.50:0/3153116302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc10e250 0x7f22bc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 --2- 172.21.15.50:0/3153116302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc105230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 --2- 172.21.15.50:0/3153116302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc105770 0x7f22bc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 -- 172.21.15.50:0/3153116302 >> 172.21.15.50:0/3153116302 conn(0x7f22bc079bd0 msgr2=0x7f22bc079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 -- 172.21.15.50:0/3153116302 shutdown_connections 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.691+0000 7f22c3e53700 1 -- 172.21.15.50:0/3153116302 wait complete. 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 Processor -- start 2024-05-17T04:33:34.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 -- start start 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 0x7f22bc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc10e250 0x7f22bc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc113c40 con 0x7f22bc104e20 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc113ac0 con 0x7f22bc10e250 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c3e53700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc113dc0 con 0x7f22bc105770 2024-05-17T04:33:34.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c1bef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c1bef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48892/0 (socket says 172.21.15.50:48892) 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c1bef700 1 -- 172.21.15.50:0/178521699 learned_addr learned my addr 172.21.15.50:0/178521699 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c23f0700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc10e250 0x7f22bc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.692+0000 7f22c13ee700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 0x7f22bc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 -- 172.21.15.50:0/178521699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 msgr2=0x7f22bc19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 0x7f22bc19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 -- 172.21.15.50:0/178521699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc10e250 msgr2=0x7f22bc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc10e250 0x7f22bc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 -- 172.21.15.50:0/178521699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc19f580 con 0x7f22bc104e20 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c13ee700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 0x7f22bc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c1bef700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006010 tx=0x7f22b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:34.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8027070 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22b800fb90 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.693+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc19f810 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.694+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b80173f0 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.694+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc19fd40 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.695+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc105230 con 0x7f22bc104e20 2024-05-17T04:33:34.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.696+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f22b802a070 con 0x7f22bc104e20 2024-05-17T04:33:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.697+0000 7f22b2ffd700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 0x7f22a80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.697+0000 7f22c13ee700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 0x7f22a80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.697+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f22b80e38b0 con 0x7f22bc104e20 2024-05-17T04:33:34.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.698+0000 7f22c13ee700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 0x7f22a80b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22bc10fe10 tx=0x7f22ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:34.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.701+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22b8013070 con 0x7f22bc104e20 2024-05-17T04:33:34.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.915+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22bc19bac0 con 0x7f22bc104e20 2024-05-17T04:33:34.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.916+0000 7f22b2ffd700 1 -- 172.21.15.50:0/178521699 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7f22b80ab110 con 0x7f22bc104e20 2024-05-17T04:33:34.918 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:33:34.918 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 msgr2=0x7f22a80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 0x7f22a80b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22bc10fe10 tx=0x7f22ac009260 comp rx=0 tx=0).stop 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 msgr2=0x7f22bc19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006010 tx=0x7f22b80057c0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 shutdown_connections 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22bc10e250 0x7f22bc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f22a80b2990 0x7f22a80b4ea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22bc104e20 0x7f22bc19a760 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 --2- 172.21.15.50:0/178521699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22bc105770 0x7f22bc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:34.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 >> 172.21.15.50:0/178521699 conn(0x7f22bc079bd0 msgr2=0x7f22bc0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:34.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 shutdown_connections 2024-05-17T04:33:34.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:34.919+0000 7f22c3e53700 1 -- 172.21.15.50:0/178521699 wait complete. 2024-05-17T04:33:35.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:34 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/178521699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:35.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:34 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/178521699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:35.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:34 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/178521699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:33:35.630 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-17T04:33:35.631 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-17T04:33:35.631 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-17T04:33:35.632 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-17T04:33:35.632 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-17T04:33:35.632 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-17T04:33:35.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-17T04:33:35.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-17T04:33:35.634 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-17T04:33:35.634 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-17T04:33:35.635 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-17T04:33:35.635 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-17T04:33:35.961 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:35 smithi050 ceph-mon[31739]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:35 smithi105 ceph-mon[34795]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:36.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:35 smithi110 ceph-mon[35497]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:36.434 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.437 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.439 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.440 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.444 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.447 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.448 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.448 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:36.834 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:38.269 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:37 smithi050 ceph-mon[31739]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:37 smithi105 ceph-mon[34795]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:37 smithi110 ceph-mon[35497]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:40.220 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:39 smithi050 ceph-mon[31739]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:39 smithi105 ceph-mon[34795]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:40.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:39 smithi110 ceph-mon[35497]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:41.205 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:41.207 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:33:42.152 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:41 smithi050 ceph-mon[31739]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 -- 172.21.15.50:0/973780936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c107790 msgr2=0x7fcd2c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 --2- 172.21.15.50:0/973780936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c107790 0x7fcd2c107ba0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0099c0 tx=0x7fcd1c009cd0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 -- 172.21.15.50:0/973780936 shutdown_connections 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 --2- 172.21.15.50:0/973780936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 --2- 172.21.15.50:0/973780936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c1034e0 0x7fcd2c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 --2- 172.21.15.50:0/973780936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c107790 0x7fcd2c107ba0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 -- 172.21.15.50:0/973780936 >> 172.21.15.50:0/973780936 conn(0x7fcd2c0fe030 msgr2=0x7fcd2c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:42.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 -- 172.21.15.50:0/973780936 shutdown_connections 2024-05-17T04:33:42.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.277+0000 7fcd335ce700 1 -- 172.21.15.50:0/973780936 wait complete. 2024-05-17T04:33:42.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 Processor -- start 2024-05-17T04:33:42.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 -- start start 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c1034e0 0x7fcd2c1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd2c107790 0x7fcd2c1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c115f10 con 0x7fcd2c1080e0 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c115d90 con 0x7fcd2c1034e0 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.278+0000 7fcd335ce700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c116090 con 0x7fcd2c107790 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34120/0 (socket says 172.21.15.50:34120) 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 -- 172.21.15.50:0/248323026 learned_addr learned my addr 172.21.15.50:0/248323026 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:42.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd3136a700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c1034e0 0x7fcd2c1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd30b69700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd2c107790 0x7fcd2c1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 -- 172.21.15.50:0/248323026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd2c107790 msgr2=0x7fcd2c1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd2c107790 0x7fcd2c1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 -- 172.21.15.50:0/248323026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c1034e0 msgr2=0x7fcd2c1a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd2c1034e0 0x7fcd2c1a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c19ce50 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd31b6b700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c19c720 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400ef30 tx=0x7fcd2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd240092e0 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd24004bb0 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c19d0e0 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004020 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.279+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1a9340 con 0x7fcd2c1080e0 2024-05-17T04:33:42.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.280+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd10001030 con 0x7fcd2c1080e0 2024-05-17T04:33:42.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.282+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcd2401b030 con 0x7fcd2c1080e0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd227fc700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcd180b2960 0x7fcd180b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcd240e2ed0 con 0x7fcd2c1080e0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd3136a700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcd180b2960 0x7fcd180b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd227fc700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] conn(0x7fcd180bdc10 0x7fcd180c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd2401ce50 con 0x7fcd180bdc10 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.283+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fcd240e3280 con 0x7fcd2c1080e0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.284+0000 7fcd30b69700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] conn(0x7fcd180bdc10 0x7fcd180c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.284+0000 7fcd3136a700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcd180b2960 0x7fcd180b4e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009680 tx=0x7fcd28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:42.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.284+0000 7fcd30b69700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] conn(0x7fcd180bdc10 0x7fcd180c0100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:42.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.285+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== osd.0 v2:172.21.15.50:6802/2573267330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd2401ce50 con 0x7fcd180bdc10 2024-05-17T04:33:42.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.301+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 --> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd10002e60 con 0x7fcd180bdc10 2024-05-17T04:33:42.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd227fc700 1 -- 172.21.15.50:0/248323026 <== osd.0 v2:172.21.15.50:6802/2573267330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd10002e60 con 0x7fcd180bdc10 2024-05-17T04:33:42.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] conn(0x7fcd180bdc10 msgr2=0x7fcd180c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] conn(0x7fcd180bdc10 0x7fcd180c0100 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcd180b2960 msgr2=0x7fcd180b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcd180b2960 0x7fcd180b4e70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009680 tx=0x7fcd28009260 comp rx=0 tx=0).stop 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 msgr2=0x7fcd2c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 --2- 172.21.15.50:0/248323026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd2c1080e0 0x7fcd2c19c720 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400ef30 tx=0x7fcd2400c3a0 comp rx=0 tx=0).stop 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd31b6b700 1 -- 172.21.15.50:0/248323026 reap_dead start 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 shutdown_connections 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.302+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 >> 172.21.15.50:0/248323026 conn(0x7fcd2c0fe030 msgr2=0x7fcd2c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.303+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 shutdown_connections 2024-05-17T04:33:42.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:42.303+0000 7fcd335ce700 1 -- 172.21.15.50:0/248323026 wait complete. 2024-05-17T04:33:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:41 smithi105 ceph-mon[34795]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:41 smithi110 ceph-mon[35497]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:43.077 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:33:43.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:33:43.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:33:44.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:43 smithi105 ceph-mon[34795]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:44.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:43 smithi110 ceph-mon[35497]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:44.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:43 smithi050 ceph-mon[31739]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:45 smithi050 ceph-mon[31739]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:45.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:45 smithi105 ceph-mon[34795]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:45.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:33:45.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:45 smithi110 ceph-mon[35497]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:48.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:47 smithi105 ceph-mon[34795]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:48.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:47 smithi110 ceph-mon[35497]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:48.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:47 smithi050 ceph-mon[31739]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:48.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 -- 172.21.15.50:0/3107279639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c102e20 msgr2=0x7fa87c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 --2- 172.21.15.50:0/3107279639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c102e20 0x7fa87c103230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa878009990 tx=0x7fa878009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 -- 172.21.15.50:0/3107279639 shutdown_connections 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 --2- 172.21.15.50:0/3107279639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa87c1080a0 0x7fa87c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 --2- 172.21.15.50:0/3107279639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c102e20 0x7fa87c103230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 --2- 172.21.15.50:0/3107279639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c107750 0x7fa87c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 -- 172.21.15.50:0/3107279639 >> 172.21.15.50:0/3107279639 conn(0x7fa87c079bd0 msgr2=0x7fa87c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 -- 172.21.15.50:0/3107279639 shutdown_connections 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.347+0000 7fa88459c700 1 -- 172.21.15.50:0/3107279639 wait complete. 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 Processor -- start 2024-05-17T04:33:48.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 -- start start 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa87c102e20 0x7fa87c105390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 0x7fa87c1058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c1080a0 0x7fa87c105e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c112bd0 con 0x7fa87c107750 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c112a50 con 0x7fa87c1080a0 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa88459c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c112d50 con 0x7fa87c102e20 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.348+0000 7fa881b37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 0x7fa87c1058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 0x7fa87c1058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34138/0 (socket says 172.21.15.50:34138) 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa882338700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa87c102e20 0x7fa87c105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 -- 172.21.15.50:0/473752744 learned_addr learned my addr 172.21.15.50:0/473752744 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa882b39700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c1080a0 0x7fa87c105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 -- 172.21.15.50:0/473752744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa87c102e20 msgr2=0x7fa87c105390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa87c102e20 0x7fa87c105390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 -- 172.21.15.50:0/473752744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c1080a0 msgr2=0x7fa87c105e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c1080a0 0x7fa87c105e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c1a8b20 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa882b39700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c1080a0 0x7fa87c105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa881b37700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 0x7fa87c1058d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d980 tx=0x7fa86c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c017070 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c1a8e10 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa86c010460 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00de60 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.349+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1a9270 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.350+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa87c063f40 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.352+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa86c010610 con 0x7fa87c107750 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa8737fe700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8680b2910 0x7fa8680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:48.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa882338700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8680b2910 0x7fa8680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa86c0e2800 con 0x7fa87c107750 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa8737fe700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] conn(0x7fa8680bdbc0 0x7fa8680c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa86c016ec0 con 0x7fa8680bdbc0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.353+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa86c0e2c50 con 0x7fa87c107750 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.354+0000 7fa882b39700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] conn(0x7fa8680bdbc0 0x7fa8680c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.354+0000 7fa882338700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8680b2910 0x7fa8680b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b5c0 tx=0x7fa878005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.354+0000 7fa882b39700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] conn(0x7fa8680bdbc0 0x7fa8680c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.355+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== osd.9 v2:172.21.15.50:6826/595367012 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa86c016ec0 con 0x7fa8680bdbc0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.372+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 --> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa87c064340 con 0x7fa8680bdbc0 2024-05-17T04:33:48.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.372+0000 7fa8737fe700 1 -- 172.21.15.50:0/473752744 <== osd.9 v2:172.21.15.50:6826/595367012 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa87c064340 con 0x7fa8680bdbc0 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] conn(0x7fa8680bdbc0 msgr2=0x7fa8680c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] conn(0x7fa8680bdbc0 0x7fa8680c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8680b2910 msgr2=0x7fa8680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8680b2910 0x7fa8680b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b5c0 tx=0x7fa878005fb0 comp rx=0 tx=0).stop 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 msgr2=0x7fa87c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 --2- 172.21.15.50:0/473752744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa87c107750 0x7fa87c1058d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d980 tx=0x7fa86c00dd40 comp rx=0 tx=0).stop 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa882b39700 1 -- 172.21.15.50:0/473752744 reap_dead start 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 shutdown_connections 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 >> 172.21.15.50:0/473752744 conn(0x7fa87c079bd0 msgr2=0x7fa87c06c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 shutdown_connections 2024-05-17T04:33:48.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:48.373+0000 7fa88459c700 1 -- 172.21.15.50:0/473752744 wait complete. 2024-05-17T04:33:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 -- 172.21.15.50:0/2960092232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d010e200 msgr2=0x7fc4d0110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/2960092232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d010e200 0x7fc4d0110710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800b0a0 tx=0x7fc4c800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 -- 172.21.15.50:0/2960092232 shutdown_connections 2024-05-17T04:33:49.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/2960092232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d010e200 0x7fc4d0110710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/2960092232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d0104da0 0x7fc4d01051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/2960092232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d01056f0 0x7fc4d010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 -- 172.21.15.50:0/2960092232 >> 172.21.15.50:0/2960092232 conn(0x7fc4d0079bd0 msgr2=0x7fc4d0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 -- 172.21.15.50:0/2960092232 shutdown_connections 2024-05-17T04:33:49.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.000+0000 7fc4d4a42700 1 -- 172.21.15.50:0/2960092232 wait complete. 2024-05-17T04:33:49.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.001+0000 7fc4d4a42700 1 Processor -- start 2024-05-17T04:33:49.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.001+0000 7fc4d4a42700 1 -- start start 2024-05-17T04:33:49.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.001+0000 7fc4d4a42700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d0104da0 0x7fc4d019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:49.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.001+0000 7fc4d4a42700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 0x7fc4d019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:49.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.001+0000 7fc4d4a42700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d010e200 0x7fc4d019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4d4a42700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0113c30 con 0x7fc4d01056f0 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4d4a42700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0113ab0 con 0x7fc4d010e200 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4d4a42700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d0113db0 con 0x7fc4d0104da0 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 0x7fc4d019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 0x7fc4d019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40474/0 (socket says 172.21.15.50:40474) 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 -- 172.21.15.50:0/854994393 learned_addr learned my addr 172.21.15.50:0/854994393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4ced9d700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d010e200 0x7fc4d019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4ce59c700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d0104da0 0x7fc4d019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 -- 172.21.15.50:0/854994393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d0104da0 msgr2=0x7fc4d019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d0104da0 0x7fc4d019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 -- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d010e200 msgr2=0x7fc4d019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4d010e200 0x7fc4d019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d019f4e0 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4ce59c700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4d0104da0 0x7fc4d019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c7fff700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 0x7fc4d019abb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d950 tx=0x7fc4b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8016070 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4b8009dd0 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800f3c0 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.002+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d019f770 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.003+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d019fc60 con 0x7fc4d01056f0 2024-05-17T04:33:49.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.003+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4d0063fc0 con 0x7fc4d01056f0 2024-05-17T04:33:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.005+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc4b8019070 con 0x7fc4d01056f0 2024-05-17T04:33:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4c77fe700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc4bc0b2990 0x7fc4bc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4ce59c700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc4bc0b2990 0x7fc4bc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:49.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc4b80e27e0 con 0x7fc4d01056f0 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4c77fe700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] conn(0x7fc4bc0bdc40 0x7fc4bc0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4bc0c0800 con 0x7fc4bc0bdc40 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.006+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc4b80e2b90 con 0x7fc4d01056f0 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.007+0000 7fc4ce59c700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc4bc0b2990 0x7fc4bc0b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005950 tx=0x7fc4c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.007+0000 7fc4ced9d700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] conn(0x7fc4bc0bdc40 0x7fc4bc0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.007+0000 7fc4ced9d700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] conn(0x7fc4bc0bdc40 0x7fc4bc0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:49.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.008+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== osd.1 v2:172.21.15.105:6800/722687301 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4bc0c0800 con 0x7fc4bc0bdc40 2024-05-17T04:33:49.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.025+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 --> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4d0064490 con 0x7fc4bc0bdc40 2024-05-17T04:33:49.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.025+0000 7fc4c77fe700 1 -- 172.21.15.50:0/854994393 <== osd.1 v2:172.21.15.105:6800/722687301 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc4d0064490 con 0x7fc4bc0bdc40 2024-05-17T04:33:49.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] conn(0x7fc4bc0bdc40 msgr2=0x7fc4bc0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] conn(0x7fc4bc0bdc40 0x7fc4bc0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc4bc0b2990 msgr2=0x7fc4bc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc4bc0b2990 0x7fc4bc0b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005950 tx=0x7fc4c00058e0 comp rx=0 tx=0).stop 2024-05-17T04:33:49.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 msgr2=0x7fc4d019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 --2- 172.21.15.50:0/854994393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4d01056f0 0x7fc4d019abb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d950 tx=0x7fc4b800dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4ced9d700 1 -- 172.21.15.50:0/854994393 reap_dead start 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 shutdown_connections 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 >> 172.21.15.50:0/854994393 conn(0x7fc4d0079bd0 msgr2=0x7fc4d00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 shutdown_connections 2024-05-17T04:33:49.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:49.026+0000 7fc4d4a42700 1 -- 172.21.15.50:0/854994393 wait complete. 2024-05-17T04:33:50.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:49 smithi105 ceph-mon[34795]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:50.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:49 smithi110 ceph-mon[35497]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:50.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:49 smithi050 ceph-mon[31739]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:33:51.020 INFO:teuthology.orchestra.run.smithi050.stdout:34359738465 2024-05-17T04:33:51.020 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 -- 172.21.15.50:0/3081642568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a301080a0 msgr2=0x7f4a3010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 --2- 172.21.15.50:0/3081642568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a301080a0 0x7f4a3010fa00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b0a0 tx=0x7f4a2800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 -- 172.21.15.50:0/3081642568 shutdown_connections 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 --2- 172.21.15.50:0/3081642568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a301080a0 0x7f4a3010fa00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 --2- 172.21.15.50:0/3081642568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a30102e70 0x7f4a30103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 --2- 172.21.15.50:0/3081642568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a30107750 0x7f4a30107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 -- 172.21.15.50:0/3081642568 >> 172.21.15.50:0/3081642568 conn(0x7f4a30079ae0 msgr2=0x7f4a30079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:51.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 -- 172.21.15.50:0/3081642568 shutdown_connections 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.252+0000 7f4a35541700 1 -- 172.21.15.50:0/3081642568 wait complete. 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 Processor -- start 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 -- start start 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 0x7f4a301a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a30107750 0x7f4a301a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a301080a0 0x7f4a3019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112b90 con 0x7f4a30102e70 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112a10 con 0x7f4a301080a0 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.253+0000 7f4a35541700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112d10 con 0x7f4a30107750 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 0x7f4a301a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 0x7f4a301a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40490/0 (socket says 172.21.15.50:40490) 2024-05-17T04:33:51.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 -- 172.21.15.50:0/2993344042 learned_addr learned my addr 172.21.15.50:0/2993344042 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a267fc700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a30107750 0x7f4a301a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2f7fe700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a301080a0 0x7f4a3019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 -- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a30107750 msgr2=0x7f4a301a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a30107750 0x7f4a301a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 -- 172.21.15.50:0/2993344042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a301080a0 msgr2=0x7f4a3019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a301080a0 0x7f4a3019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3019cd90 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2effd700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 0x7f4a301a24d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000d950 tx=0x7f4a2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20016070 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3019d020 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a2000fd20 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000e3b0 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.254+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a301a9280 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.255+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a3010f9d0 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.257+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4a20019070 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a2cff9700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4a1c0b2990 0x7f4a1c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a267fc700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4a1c0b2990 0x7f4a1c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4a200e2840 con 0x7f4a30102e70 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a2cff9700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] conn(0x7f4a1c0bdc40 0x7f4a1c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:51.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a20020e70 con 0x7f4a1c0bdc40 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4a200e2bf0 con 0x7f4a30102e70 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.258+0000 7f4a267fc700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4a1c0b2990 0x7f4a1c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a3019de00 tx=0x7f4a1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.259+0000 7f4a2f7fe700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] conn(0x7f4a1c0bdc40 0x7f4a1c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.259+0000 7f4a2f7fe700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] conn(0x7f4a1c0bdc40 0x7f4a1c0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.260+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== osd.5 v2:172.21.15.110:6808/1542934887 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a20020e70 con 0x7f4a1c0bdc40 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.276+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 --> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a301042d0 con 0x7f4a1c0bdc40 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.277+0000 7f4a2cff9700 1 -- 172.21.15.50:0/2993344042 <== osd.5 v2:172.21.15.110:6808/1542934887 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a301042d0 con 0x7f4a1c0bdc40 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.277+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] conn(0x7f4a1c0bdc40 msgr2=0x7f4a1c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.277+0000 7f4a35541700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] conn(0x7f4a1c0bdc40 0x7f4a1c0c0130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4a1c0b2990 msgr2=0x7f4a1c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4a1c0b2990 0x7f4a1c0b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a3019de00 tx=0x7f4a1800b410 comp rx=0 tx=0).stop 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 msgr2=0x7f4a301a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:51.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 --2- 172.21.15.50:0/2993344042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a30102e70 0x7f4a301a24d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000d950 tx=0x7f4a2000dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:51.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a2f7fe700 1 -- 172.21.15.50:0/2993344042 reap_dead start 2024-05-17T04:33:51.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 shutdown_connections 2024-05-17T04:33:51.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 >> 172.21.15.50:0/2993344042 conn(0x7f4a30079ae0 msgr2=0x7f4a30104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:51.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 shutdown_connections 2024-05-17T04:33:51.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:51.278+0000 7f4a35541700 1 -- 172.21.15.50:0/2993344042 wait complete. 2024-05-17T04:33:51.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:51 smithi050 ceph-mon[31739]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:52.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:51 smithi105 ceph-mon[34795]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:52.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:51 smithi110 ceph-mon[35497]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 -- 172.21.15.50:0/1437191342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78108080 msgr2=0x7f1b7810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1437191342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78108080 0x7f1b7810f9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000b0a0 tx=0x7f1b7000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 -- 172.21.15.50:0/1437191342 shutdown_connections 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1437191342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78108080 0x7f1b7810f9e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1437191342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78102e00 0x7f1b78103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1437191342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b78107730 0x7f1b78107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 -- 172.21.15.50:0/1437191342 >> 172.21.15.50:0/1437191342 conn(0x7f1b78079bd0 msgr2=0x7f1b78079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 -- 172.21.15.50:0/1437191342 shutdown_connections 2024-05-17T04:33:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.116+0000 7f1b7d240700 1 -- 172.21.15.50:0/1437191342 wait complete. 2024-05-17T04:33:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 Processor -- start 2024-05-17T04:33:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 -- start start 2024-05-17T04:33:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78102e00 0x7f1b7819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 0x7f1b7819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b78108080 0x7f1b7819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78112b70 con 0x7f1b78107730 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781129f0 con 0x7f1b78108080 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7d240700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78112cf0 con 0x7f1b78102e00 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 0x7f1b7819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 0x7f1b7819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40514/0 (socket says 172.21.15.50:40514) 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7659c700 1 -- 172.21.15.50:0/1250599377 learned_addr learned my addr 172.21.15.50:0/1250599377 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b7759e700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b78108080 0x7f1b7819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.117+0000 7f1b76d9d700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78102e00 0x7f1b7819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 -- 172.21.15.50:0/1250599377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78102e00 msgr2=0x7f1b7819b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78102e00 0x7f1b7819b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 -- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b78108080 msgr2=0x7f1b7819bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b78108080 0x7f1b7819bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b781a6810 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7659c700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 0x7f1b7819ba00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e970 tx=0x7f1b6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c004a30 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b781a6aa0 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c00cc70 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c005520 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.118+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b781a7030 con 0x7f1b78107730 2024-05-17T04:33:52.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.119+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b64001030 con 0x7f1b78107730 2024-05-17T04:33:52.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.121+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1b6c004bd0 con 0x7f1b78107730 2024-05-17T04:33:52.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.121+0000 7f1b5ffff700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1b600b2990 0x7f1b600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:52.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b76d9d700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1b600b2990 0x7f1b600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1b6c0e2d00 con 0x7f1b78107730 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b5ffff700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] conn(0x7f1b600bdc40 0x7f1b600c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b6c011e50 con 0x7f1b600bdc40 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c0e30b0 con 0x7f1b78107730 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b76d9d700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1b600b2990 0x7f1b600b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b68005950 tx=0x7f1b6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:52.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.122+0000 7f1b7759e700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] conn(0x7f1b600bdc40 0x7f1b600c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:52.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.123+0000 7f1b7759e700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] conn(0x7f1b600bdc40 0x7f1b600c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:52.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.124+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== osd.4 v2:172.21.15.105:6808/908958206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b6c011e50 con 0x7f1b600bdc40 2024-05-17T04:33:52.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.140+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 --> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b64002e60 con 0x7f1b600bdc40 2024-05-17T04:33:52.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b5ffff700 1 -- 172.21.15.50:0/1250599377 <== osd.4 v2:172.21.15.105:6808/908958206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b64002e60 con 0x7f1b600bdc40 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] conn(0x7f1b600bdc40 msgr2=0x7f1b600c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] conn(0x7f1b600bdc40 0x7f1b600c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1b600b2990 msgr2=0x7f1b600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1b600b2990 0x7f1b600b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b68005950 tx=0x7f1b6800b410 comp rx=0 tx=0).stop 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 msgr2=0x7f1b7819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 --2- 172.21.15.50:0/1250599377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b78107730 0x7f1b7819ba00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e970 tx=0x7f1b6c00ec80 comp rx=0 tx=0).stop 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7759e700 1 -- 172.21.15.50:0/1250599377 reap_dead start 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 shutdown_connections 2024-05-17T04:33:52.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.141+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 >> 172.21.15.50:0/1250599377 conn(0x7f1b78079bd0 msgr2=0x7f1b781057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:52.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.142+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 shutdown_connections 2024-05-17T04:33:52.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:52.142+0000 7f1b7d240700 1 -- 172.21.15.50:0/1250599377 wait complete. 2024-05-17T04:33:53.232 INFO:teuthology.orchestra.run.smithi050.stdout:55834574941 2024-05-17T04:33:53.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-17T04:33:53.973 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:53 smithi050 ceph-mon[31739]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:54.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:53 smithi105 ceph-mon[34795]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:54.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:53 smithi110 ceph-mon[35497]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.426+0000 7f46263a7700 1 -- 172.21.15.50:0/1258818368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46201080a0 msgr2=0x7f462010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.426+0000 7f46263a7700 1 --2- 172.21.15.50:0/1258818368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46201080a0 0x7f462010fa00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f461800b0a0 tx=0x7f461800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 -- 172.21.15.50:0/1258818368 shutdown_connections 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 --2- 172.21.15.50:0/1258818368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46201080a0 0x7f462010fa00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 --2- 172.21.15.50:0/1258818368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4620102e70 0x7f4620103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 --2- 172.21.15.50:0/1258818368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4620107750 0x7f4620107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 -- 172.21.15.50:0/1258818368 >> 172.21.15.50:0/1258818368 conn(0x7f4620079ae0 msgr2=0x7f4620079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:55.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 -- 172.21.15.50:0/1258818368 shutdown_connections 2024-05-17T04:33:55.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 -- 172.21.15.50:0/1258818368 wait complete. 2024-05-17T04:33:55.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 Processor -- start 2024-05-17T04:33:55.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.427+0000 7f46263a7700 1 -- start start 2024-05-17T04:33:55.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4620102e70 0x7f462019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:55.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 0x7f462019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46201080a0 0x7f462019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4620112be0 con 0x7f4620107750 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4620112a60 con 0x7f4620102e70 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f46263a7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4620112d60 con 0x7f46201080a0 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 0x7f462019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 0x7f462019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40536/0 (socket says 172.21.15.50:40536) 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 -- 172.21.15.50:0/581645818 learned_addr learned my addr 172.21.15.50:0/581645818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f4624944700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46201080a0 0x7f462019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461ffff700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4620102e70 0x7f462019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 -- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46201080a0 msgr2=0x7f462019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46201080a0 0x7f462019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 -- 172.21.15.50:0/581645818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4620102e70 msgr2=0x7f462019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4620102e70 0x7f462019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.428+0000 7f461f7fe700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462019f510 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f461f7fe700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 0x7f462019ac90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f461400c9e0 tx=0x7f461400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614007aa0 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4614007c40 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462019f7a0 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614018400 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.429+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462019fce0 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.430+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f462006c390 con 0x7f4620107750 2024-05-17T04:33:55.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.432+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f461400f450 con 0x7f4620107750 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.432+0000 7f461d7fa700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46080b2910 0x7f46080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461ffff700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46080b2910 0x7f46080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f46140e2a10 con 0x7f4620107750 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461d7fa700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] conn(0x7f46080bdbc0 0x7f46080c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f461401fe50 con 0x7f46080bdbc0 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f46140e2e60 con 0x7f4620107750 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f461ffff700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46080b2910 0x7f46080b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4610007be0 tx=0x7f46100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:55.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.433+0000 7f4624944700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] conn(0x7f46080bdbc0 0x7f46080c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:55.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.434+0000 7f4624944700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] conn(0x7f46080bdbc0 0x7f46080c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:55.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.434+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== osd.2 v2:172.21.15.110:6800/3787373914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f461401fe50 con 0x7f46080bdbc0 2024-05-17T04:33:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.451+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 --> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46201042d0 con 0x7f46080bdbc0 2024-05-17T04:33:55.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.451+0000 7f461d7fa700 1 -- 172.21.15.50:0/581645818 <== osd.2 v2:172.21.15.110:6800/3787373914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46201042d0 con 0x7f46080bdbc0 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] conn(0x7f46080bdbc0 msgr2=0x7f46080c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] conn(0x7f46080bdbc0 0x7f46080c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46080b2910 msgr2=0x7f46080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f46080b2910 0x7f46080b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4610007be0 tx=0x7f46100073d0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 msgr2=0x7f462019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:55.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 --2- 172.21.15.50:0/581645818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4620107750 0x7f462019ac90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f461400c9e0 tx=0x7f461400ccf0 comp rx=0 tx=0).stop 2024-05-17T04:33:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f4624944700 1 -- 172.21.15.50:0/581645818 reap_dead start 2024-05-17T04:33:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 shutdown_connections 2024-05-17T04:33:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 >> 172.21.15.50:0/581645818 conn(0x7f4620079ae0 msgr2=0x7f4620101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 shutdown_connections 2024-05-17T04:33:55.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:55.452+0000 7f46263a7700 1 -- 172.21.15.50:0/581645818 wait complete. 2024-05-17T04:33:55.916 INFO:teuthology.orchestra.run.smithi050.stdout:133143986254 2024-05-17T04:33:55.917 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-17T04:33:56.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:55 smithi105 ceph-mon[34795]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:56.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:55 smithi110 ceph-mon[35497]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:56.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:55 smithi050 ceph-mon[31739]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:33:56.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 -- 172.21.15.50:0/2121963161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0548107760 msgr2=0x7f0548107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 --2- 172.21.15.50:0/2121963161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0548107760 0x7f0548107b70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f053c009990 tx=0x7f053c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 -- 172.21.15.50:0/2121963161 shutdown_connections 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 --2- 172.21.15.50:0/2121963161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05481080b0 0x7f054810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 --2- 172.21.15.50:0/2121963161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 0x7f05481038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 --2- 172.21.15.50:0/2121963161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0548107760 0x7f0548107b70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 -- 172.21.15.50:0/2121963161 >> 172.21.15.50:0/2121963161 conn(0x7f0548078040 msgr2=0x7f05480784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 -- 172.21.15.50:0/2121963161 shutdown_connections 2024-05-17T04:33:56.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.355+0000 7f054e1c8700 1 -- 172.21.15.50:0/2121963161 wait complete. 2024-05-17T04:33:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.356+0000 7f054e1c8700 1 Processor -- start 2024-05-17T04:33:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.356+0000 7f054e1c8700 1 -- start start 2024-05-17T04:33:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 0x7f054819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0548107760 0x7f054819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:56.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05481080b0 0x7f054819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0548107760 0x7f054819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f0547fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05481080b0 0x7f054819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0548112c00 con 0x7f05481080b0 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f05477fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 0x7f054819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0548107760 0x7f054819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:45386/0 (socket says 172.21.15.50:45386) 2024-05-17T04:33:56.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0548112a80 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f0546ffd700 1 -- 172.21.15.50:0/805138128 learned_addr learned my addr 172.21.15.50:0/805138128 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0548112d80 con 0x7f0548107760 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f05477fe700 1 -- 172.21.15.50:0/805138128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0548107760 msgr2=0x7f054819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f05477fe700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0548107760 0x7f054819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f05477fe700 1 -- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05481080b0 msgr2=0x7f054819c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.357+0000 7f05477fe700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05481080b0 0x7f054819c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f05477fe700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05481a6960 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f0547fff700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05481080b0 0x7f054819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f05477fe700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 0x7f054819b610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9c0 tx=0x7f053800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538004220 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0548199460 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05380043c0 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538003b40 con 0x7f05481034b0 2024-05-17T04:33:56.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.358+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05481999a0 con 0x7f05481034b0 2024-05-17T04:33:56.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.359+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0548063f40 con 0x7f05481034b0 2024-05-17T04:33:56.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.361+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0538003ce0 con 0x7f05481034b0 2024-05-17T04:33:56.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0544ff9700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f05300b2910 0x7f05300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:56.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f05380e2db0 con 0x7f05481034b0 2024-05-17T04:33:56.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0546ffd700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f05300b2910 0x7f05300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0544ff9700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] conn(0x7f05300bdbc0 0x7f05300c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0538021e50 con 0x7f05300bdbc0 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.362+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f05380e3200 con 0x7f05481034b0 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.363+0000 7f0547fff700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] conn(0x7f05300bdbc0 0x7f05300c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.363+0000 7f0546ffd700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f05300b2910 0x7f05300b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f053c009750 tx=0x7f053c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:56.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.363+0000 7f0547fff700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] conn(0x7f05300bdbc0 0x7f05300c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:56.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.364+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== osd.6 v2:172.21.15.50:6818/159814107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0538021e50 con 0x7f05300bdbc0 2024-05-17T04:33:56.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.381+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 --> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f05480643c0 con 0x7f05300bdbc0 2024-05-17T04:33:56.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f0544ff9700 1 -- 172.21.15.50:0/805138128 <== osd.6 v2:172.21.15.50:6818/159814107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f05480643c0 con 0x7f05300bdbc0 2024-05-17T04:33:56.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] conn(0x7f05300bdbc0 msgr2=0x7f05300c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] conn(0x7f05300bdbc0 0x7f05300c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f05300b2910 msgr2=0x7f05300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f05300b2910 0x7f05300b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f053c009750 tx=0x7f053c00b260 comp rx=0 tx=0).stop 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 msgr2=0x7f054819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f054e1c8700 1 --2- 172.21.15.50:0/805138128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05481034b0 0x7f054819b610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9c0 tx=0x7f053800ecd0 comp rx=0 tx=0).stop 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.382+0000 7f0547fff700 1 -- 172.21.15.50:0/805138128 reap_dead start 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.383+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 shutdown_connections 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.383+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 >> 172.21.15.50:0/805138128 conn(0x7f0548078040 msgr2=0x7f05480ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.383+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 shutdown_connections 2024-05-17T04:33:56.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:56.383+0000 7f054e1c8700 1 -- 172.21.15.50:0/805138128 wait complete. 2024-05-17T04:33:56.830 INFO:teuthology.orchestra.run.smithi050.stdout:253403070516 2024-05-17T04:33:56.830 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-17T04:33:57.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 -- 172.21.15.50:0/2971502996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8109190 msgr2=0x7f7af8110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 --2- 172.21.15.50:0/2971502996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8109190 0x7f7af8110af0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7af000b0a0 tx=0x7f7af000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 -- 172.21.15.50:0/2971502996 shutdown_connections 2024-05-17T04:33:57.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 --2- 172.21.15.50:0/2971502996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8109190 0x7f7af8110af0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 --2- 172.21.15.50:0/2971502996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af8103f10 0x7f7af8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 --2- 172.21.15.50:0/2971502996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af8108840 0x7f7af8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 -- 172.21.15.50:0/2971502996 >> 172.21.15.50:0/2971502996 conn(0x7f7af80ff9b0 msgr2=0x7f7af8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:57.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 -- 172.21.15.50:0/2971502996 shutdown_connections 2024-05-17T04:33:57.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.496+0000 7f7afd314700 1 -- 172.21.15.50:0/2971502996 wait complete. 2024-05-17T04:33:57.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 Processor -- start 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 -- start start 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af8103f10 0x7f7af819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 0x7f7af819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af8109190 0x7f7af819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8113c90 con 0x7f7af8108840 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8113b10 con 0x7f7af8109190 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.497+0000 7f7afd314700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8113e10 con 0x7f7af8103f10 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 0x7f7af819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 0x7f7af819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40602/0 (socket says 172.21.15.50:40602) 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 -- 172.21.15.50:0/921851526 learned_addr learned my addr 172.21.15.50:0/921851526 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af6ffd700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af8103f10 0x7f7af819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:57.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af77fe700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af8109190 0x7f7af819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 -- 172.21.15.50:0/921851526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af8103f10 msgr2=0x7f7af819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af8103f10 0x7f7af819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 -- 172.21.15.50:0/921851526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af8109190 msgr2=0x7f7af819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7af8109190 0x7f7af819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819f5e0 con 0x7f7af8108840 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7af67fc700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 0x7f7af819acb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d950 tx=0x7f7aec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec017070 con 0x7f7af8108840 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7aec005020 con 0x7f7af8108840 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819f870 con 0x7f7af8108840 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00f980 con 0x7f7af8108840 2024-05-17T04:33:57.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.498+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af819fd60 con 0x7f7af8108840 2024-05-17T04:33:57.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.499+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7af8063f00 con 0x7f7af8108840 2024-05-17T04:33:57.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.501+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7aec004a30 con 0x7f7af8108840 2024-05-17T04:33:57.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7adffff700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ae00b2910 0x7f7ae00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:57.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7af6ffd700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ae00b2910 0x7f7ae00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:57.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7aec0e2a20 con 0x7f7af8108840 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7adffff700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] conn(0x7f7ae00bdbc0 0x7f7ae00c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ae00c0780 con 0x7f7ae00bdbc0 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7aec0e2e70 con 0x7f7af8108840 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.502+0000 7f7af77fe700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] conn(0x7f7ae00bdbc0 0x7f7ae00c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.503+0000 7f7af6ffd700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ae00b2910 0x7f7ae00b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009c60 tx=0x7f7ae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.503+0000 7f7af77fe700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] conn(0x7f7ae00bdbc0 0x7f7ae00c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:57.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.504+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== osd.3 v2:172.21.15.50:6810/3258879203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ae00c0780 con 0x7f7ae00bdbc0 2024-05-17T04:33:57.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.520+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 --> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7af8064300 con 0x7f7ae00bdbc0 2024-05-17T04:33:57.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.520+0000 7f7adffff700 1 -- 172.21.15.50:0/921851526 <== osd.3 v2:172.21.15.50:6810/3258879203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7af8064300 con 0x7f7ae00bdbc0 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] conn(0x7f7ae00bdbc0 msgr2=0x7f7ae00c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] conn(0x7f7ae00bdbc0 0x7f7ae00c00b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ae00b2910 msgr2=0x7f7ae00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ae00b2910 0x7f7ae00b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009c60 tx=0x7f7ae8009260 comp rx=0 tx=0).stop 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 msgr2=0x7f7af819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 --2- 172.21.15.50:0/921851526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7af8108840 0x7f7af819acb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d950 tx=0x7f7aec00dc60 comp rx=0 tx=0).stop 2024-05-17T04:33:57.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7af77fe700 1 -- 172.21.15.50:0/921851526 reap_dead start 2024-05-17T04:33:57.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 shutdown_connections 2024-05-17T04:33:57.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 >> 172.21.15.50:0/921851526 conn(0x7f7af80ff9b0 msgr2=0x7f7af8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:57.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 shutdown_connections 2024-05-17T04:33:57.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.521+0000 7f7afd314700 1 -- 172.21.15.50:0/921851526 wait complete. 2024-05-17T04:33:58.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:57 smithi105 ceph-mon[34795]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:58.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:57 smithi110 ceph-mon[35497]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:58.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:57 smithi050 ceph-mon[31739]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-17T04:33:58.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.038+0000 7f606e5ef700 1 -- 172.21.15.50:0/3090083904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068107730 msgr2=0x7f6068107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3090083904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068107730 0x7f6068107b40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f605c009990 tx=0x7f605c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 -- 172.21.15.50:0/3090083904 shutdown_connections 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3090083904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6068108080 0x7f606810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3090083904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6068102e00 0x7f6068103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3090083904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068107730 0x7f6068107b40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 -- 172.21.15.50:0/3090083904 >> 172.21.15.50:0/3090083904 conn(0x7f6068079bd0 msgr2=0x7f6068079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 -- 172.21.15.50:0/3090083904 shutdown_connections 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 -- 172.21.15.50:0/3090083904 wait complete. 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 Processor -- start 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.039+0000 7f606e5ef700 1 -- start start 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6068102e00 0x7f606819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6068107730 0x7f606819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 0x7f606819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6068112b40 con 0x7f6068108080 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60681129c0 con 0x7f6068102e00 2024-05-17T04:33:58.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606e5ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6068112cc0 con 0x7f6068107730 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 0x7f606819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 0x7f606819bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40578/0 (socket says 172.21.15.50:40578) 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 -- 172.21.15.50:0/3441060944 learned_addr learned my addr 172.21.15.50:0/3441060944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f6067fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6068102e00 0x7f606819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f60677fe700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6068107730 0x7f606819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 -- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6068107730 msgr2=0x7f606819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6068107730 0x7f606819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 -- 172.21.15.50:0/3441060944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6068102e00 msgr2=0x7f606819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6068102e00 0x7f606819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.040+0000 7f606cb8c700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60681a68a0 con 0x7f6068108080 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f606cb8c700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 0x7f606819bff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6060007ee0 tx=0x7f606000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000dd20 con 0x7f6068108080 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60681a6b90 con 0x7f6068108080 2024-05-17T04:33:58.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6060004020 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060015660 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.041+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60681a70a0 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.042+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6068063f40 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.044+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f60600041d0 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.044+0000 7f60657fa700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60500b2910 0x7f60500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f6067fff700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60500b2910 0x7f60500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f60600e2bd0 con 0x7f6068108080 2024-05-17T04:33:58.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f60657fa700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] conn(0x7f60500bdbc0 0x7f60500c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60500c0780 con 0x7f60500bdbc0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f60600e3020 con 0x7f6068108080 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f60677fe700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] conn(0x7f60500bdbc0 0x7f60500c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.045+0000 7f6067fff700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60500b2910 0x7f60500b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6058009c60 tx=0x7f6058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.046+0000 7f60677fe700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] conn(0x7f60500bdbc0 0x7f60500c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.047+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== osd.11 v2:172.21.15.110:6824/148686572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60500c0780 con 0x7f60500bdbc0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 --> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6068064340 con 0x7f60500bdbc0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f60657fa700 1 -- 172.21.15.50:0/3441060944 <== osd.11 v2:172.21.15.110:6824/148686572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6068064340 con 0x7f60500bdbc0 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] conn(0x7f60500bdbc0 msgr2=0x7f60500c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] conn(0x7f60500bdbc0 0x7f60500c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60500b2910 msgr2=0x7f60500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60500b2910 0x7f60500b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6058009c60 tx=0x7f6058009260 comp rx=0 tx=0).stop 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 msgr2=0x7f606819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606e5ef700 1 --2- 172.21.15.50:0/3441060944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6068108080 0x7f606819bff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6060007ee0 tx=0x7f606000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.064+0000 7f606cb8c700 1 -- 172.21.15.50:0/3441060944 reap_dead start 2024-05-17T04:33:58.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.065+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 shutdown_connections 2024-05-17T04:33:58.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.065+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 >> 172.21.15.50:0/3441060944 conn(0x7f6068079bd0 msgr2=0x7f6068105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:33:58.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.065+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 shutdown_connections 2024-05-17T04:33:58.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:33:57.065+0000 7f606e5ef700 1 -- 172.21.15.50:0/3441060944 wait complete. 2024-05-17T04:33:58.823 INFO:teuthology.orchestra.run.smithi050.stdout:154618822730 2024-05-17T04:33:58.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-17T04:33:59.274 INFO:teuthology.orchestra.run.smithi050.stdout:180388626502 2024-05-17T04:33:59.275 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-17T04:33:59.738 INFO:teuthology.orchestra.run.smithi050.stdout:77309411417 2024-05-17T04:33:59.738 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-17T04:33:59.960 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:33:59 smithi050 ceph-mon[31739]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T04:34:00.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:33:59 smithi105 ceph-mon[34795]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T04:34:00.069 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:33:59 smithi110 ceph-mon[35497]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-17T04:34:00.884 INFO:teuthology.orchestra.run.smithi050.stdout:107374182485 2024-05-17T04:34:00.884 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-17T04:34:01.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 -- 172.21.15.50:0/1428152145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc1032e0 msgr2=0x7f17fc1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 --2- 172.21.15.50:0/1428152145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc1032e0 0x7f17fc1036f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009990 tx=0x7f17ec009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 -- 172.21.15.50:0/1428152145 shutdown_connections 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 --2- 172.21.15.50:0/1428152145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc103c30 0x7f17fc10f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 --2- 172.21.15.50:0/1428152145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc107b80 0x7f17fc107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 --2- 172.21.15.50:0/1428152145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc1032e0 0x7f17fc1036f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.138+0000 7f18047ea700 1 -- 172.21.15.50:0/1428152145 >> 172.21.15.50:0/1428152145 conn(0x7f17fc0780d0 msgr2=0x7f17fc078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.139+0000 7f18047ea700 1 -- 172.21.15.50:0/1428152145 shutdown_connections 2024-05-17T04:34:01.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.139+0000 7f18047ea700 1 -- 172.21.15.50:0/1428152145 wait complete. 2024-05-17T04:34:01.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.140+0000 7f18047ea700 1 Processor -- start 2024-05-17T04:34:01.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.140+0000 7f18047ea700 1 -- start start 2024-05-17T04:34:01.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.140+0000 7f18047ea700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 0x7f17fc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f18047ea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 0x7f17fc1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f18047ea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 0x7f17fc197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f1802586700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 0x7f17fc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f1801d85700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 0x7f17fc1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f1802586700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 0x7f17fc19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40202/0 (socket says 172.21.15.50:40202) 2024-05-17T04:34:01.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f1801d85700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 0x7f17fc1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33870/0 (socket says 172.21.15.50:33870) 2024-05-17T04:34:01.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.141+0000 7f1802d87700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 0x7f17fc197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f1802d87700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 0x7f17fc197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50968/0 (socket says 172.21.15.50:50968) 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f1802586700 1 -- 172.21.15.50:0/3367772655 learned_addr learned my addr 172.21.15.50:0/3367772655 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111b80 con 0x7f17fc107b80 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111a00 con 0x7f17fc1032e0 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111d00 con 0x7f17fc103c30 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f1802586700 1 -- 172.21.15.50:0/3367772655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 msgr2=0x7f17fc1970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f1802586700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 0x7f17fc1970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.142+0000 7f1802586700 1 -- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 msgr2=0x7f17fc197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f1802586700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 0x7f17fc197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f1802586700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc197d00 con 0x7f17fc1032e0 2024-05-17T04:34:01.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f1801d85700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17fc103c30 0x7f17fc1970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:01.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f1802d87700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17fc107b80 0x7f17fc197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:01.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f1802586700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 0x7f17fc19f050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e970 tx=0x7f17f800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8004220 con 0x7f17fc1032e0 2024-05-17T04:34:01.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17f80043c0 con 0x7f17fc1032e0 2024-05-17T04:34:01.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8003b10 con 0x7f17fc1032e0 2024-05-17T04:34:01.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.143+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1b5ac0 con 0x7f17fc1032e0 2024-05-17T04:34:01.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.144+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1b5e60 con 0x7f17fc1032e0 2024-05-17T04:34:01.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.144+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17fc063fc0 con 0x7f17fc1032e0 2024-05-17T04:34:01.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.146+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f17f8003cf0 con 0x7f17fc1032e0 2024-05-17T04:34:01.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f17f37fe700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f17e80b2a40 0x7f17e80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f1801d85700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f17e80b2a40 0x7f17e80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f17f80e2d60 con 0x7f17fc1032e0 2024-05-17T04:34:01.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f17f37fe700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] conn(0x7f17e80bdcf0 0x7f17e80c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17f801aec0 con 0x7f17e80bdcf0 2024-05-17T04:34:01.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.147+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f17f80e3150 con 0x7f17fc1032e0 2024-05-17T04:34:01.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.148+0000 7f1802d87700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] conn(0x7f17e80bdcf0 0x7f17e80c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.148+0000 7f1801d85700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f17e80b2a40 0x7f17e80b4f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006010 tx=0x7f17ec009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.148+0000 7f1802d87700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] conn(0x7f17e80bdcf0 0x7f17e80c01e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.149+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== osd.10 v2:172.21.15.105:6824/3101480579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17f801aec0 con 0x7f17e80bdcf0 2024-05-17T04:34:01.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.167+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 --> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17fc064440 con 0x7f17e80bdcf0 2024-05-17T04:34:01.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.167+0000 7f17f37fe700 1 -- 172.21.15.50:0/3367772655 <== osd.10 v2:172.21.15.105:6824/3101480579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f17fc064440 con 0x7f17e80bdcf0 2024-05-17T04:34:01.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.167+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] conn(0x7f17e80bdcf0 msgr2=0x7f17e80c01e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.167+0000 7f18047ea700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] conn(0x7f17e80bdcf0 0x7f17e80c01e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f17e80b2a40 msgr2=0x7f17e80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f17e80b2a40 0x7f17e80b4f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006010 tx=0x7f17ec009510 comp rx=0 tx=0).stop 2024-05-17T04:34:01.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 msgr2=0x7f17fc19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 --2- 172.21.15.50:0/3367772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17fc1032e0 0x7f17fc19f050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17f800e970 tx=0x7f17f800ec80 comp rx=0 tx=0).stop 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f1802d87700 1 -- 172.21.15.50:0/3367772655 reap_dead start 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 shutdown_connections 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 >> 172.21.15.50:0/3367772655 conn(0x7f17fc0780d0 msgr2=0x7f17fc100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 shutdown_connections 2024-05-17T04:34:01.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.168+0000 7f18047ea700 1 -- 172.21.15.50:0/3367772655 wait complete. 2024-05-17T04:34:01.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 -- 172.21.15.50:0/2551810008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be410e270 msgr2=0x7f5be4110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 --2- 172.21.15.50:0/2551810008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be410e270 0x7f5be4110740 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00b0a0 tx=0x7f5bdc00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 -- 172.21.15.50:0/2551810008 shutdown_connections 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 --2- 172.21.15.50:0/2551810008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be410e270 0x7f5be4110740 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 --2- 172.21.15.50:0/2551810008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5be4104e90 0x7f5be41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 --2- 172.21.15.50:0/2551810008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be41057e0 0x7f5be410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 -- 172.21.15.50:0/2551810008 >> 172.21.15.50:0/2551810008 conn(0x7f5be4079ae0 msgr2=0x7f5be4079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 -- 172.21.15.50:0/2551810008 shutdown_connections 2024-05-17T04:34:01.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.881+0000 7f5bec453700 1 -- 172.21.15.50:0/2551810008 wait complete. 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 Processor -- start 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 -- start start 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 0x7f5be419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5be41057e0 0x7f5be419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be410e270 0x7f5be419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113cb0 con 0x7f5be4104e90 2024-05-17T04:34:01.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113b30 con 0x7f5be410e270 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.882+0000 7f5bec453700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113e30 con 0x7f5be41057e0 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 0x7f5be419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea9f0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be410e270 0x7f5be419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 0x7f5be419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50978/0 (socket says 172.21.15.50:50978) 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 -- 172.21.15.50:0/2021362979 learned_addr learned my addr 172.21.15.50:0/2021362979 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5be99ee700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5be41057e0 0x7f5be419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 -- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5be41057e0 msgr2=0x7f5be419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5be41057e0 0x7f5be419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 -- 172.21.15.50:0/2021362979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be410e270 msgr2=0x7f5be419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be410e270 0x7f5be419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be419f580 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea9f0700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be410e270 0x7f5be419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.883+0000 7f5bea1ef700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 0x7f5be419a6b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5be000d950 tx=0x7f5be000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.884+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0016070 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.884+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5be000fd20 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.884+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be419f810 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.884+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000e3b0 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.884+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a7190 con 0x7f5be4104e90 2024-05-17T04:34:01.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.885+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5be4063fc0 con 0x7f5be4104e90 2024-05-17T04:34:01.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.887+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5be0019070 con 0x7f5be4104e90 2024-05-17T04:34:01.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.887+0000 7f5bdb7fe700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5bd00b2990 0x7f5bd00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5be99ee700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5bd00b2990 0x7f5bd00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5be00e2610 con 0x7f5be4104e90 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5bdb7fe700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] conn(0x7f5bd00bdc40 0x7f5bd00c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bd00c0800 con 0x7f5bd00bdc40 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5be00e2a60 con 0x7f5be4104e90 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5be99ee700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5bd00b2990 0x7f5bd00b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5be4102400 tx=0x7f5bd400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.888+0000 7f5bea9f0700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] conn(0x7f5bd00bdc40 0x7f5bd00c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:01.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.889+0000 7f5bea9f0700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] conn(0x7f5bd00bdc40 0x7f5bd00c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:01.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.890+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== osd.8 v2:172.21.15.110:6816/2523141158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd00c0800 con 0x7f5bd00bdc40 2024-05-17T04:34:01.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.911+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 --> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5be40643c0 con 0x7f5bd00bdc40 2024-05-17T04:34:01.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bdb7fe700 1 -- 172.21.15.50:0/2021362979 <== osd.8 v2:172.21.15.110:6816/2523141158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5be40643c0 con 0x7f5bd00bdc40 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] conn(0x7f5bd00bdc40 msgr2=0x7f5bd00c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] conn(0x7f5bd00bdc40 0x7f5bd00c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5bd00b2990 msgr2=0x7f5bd00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5bd00b2990 0x7f5bd00b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5be4102400 tx=0x7f5bd400b410 comp rx=0 tx=0).stop 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 msgr2=0x7f5be419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 --2- 172.21.15.50:0/2021362979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be4104e90 0x7f5be419a6b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5be000d950 tx=0x7f5be000dc60 comp rx=0 tx=0).stop 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bea9f0700 1 -- 172.21.15.50:0/2021362979 reap_dead start 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 shutdown_connections 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.912+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 >> 172.21.15.50:0/2021362979 conn(0x7f5be4079ae0 msgr2=0x7f5be40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:01.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.913+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 shutdown_connections 2024-05-17T04:34:01.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:01.913+0000 7f5bec453700 1 -- 172.21.15.50:0/2021362979 wait complete. 2024-05-17T04:34:02.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 -- 172.21.15.50:0/3340381678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830109190 msgr2=0x7fa830110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 --2- 172.21.15.50:0/3340381678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830109190 0x7fa830110af0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b0a0 tx=0x7fa82800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 -- 172.21.15.50:0/3340381678 shutdown_connections 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 --2- 172.21.15.50:0/3340381678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830109190 0x7fa830110af0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 --2- 172.21.15.50:0/3340381678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa830103f10 0x7fa830104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 --2- 172.21.15.50:0/3340381678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa830108840 0x7fa830108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.498+0000 7fa83759e700 1 -- 172.21.15.50:0/3340381678 >> 172.21.15.50:0/3340381678 conn(0x7fa8300ff9b0 msgr2=0x7fa830101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.499+0000 7fa83759e700 1 -- 172.21.15.50:0/3340381678 shutdown_connections 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.499+0000 7fa83759e700 1 -- 172.21.15.50:0/3340381678 wait complete. 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.499+0000 7fa83759e700 1 Processor -- start 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.499+0000 7fa83759e700 1 -- start start 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 0x7fa83019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa830108840 0x7fa83019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa830109190 0x7fa83019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:02.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa830113c80 con 0x7fa830103f10 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa830113b00 con 0x7fa830108840 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83759e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa830113e00 con 0x7fa830109190 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 0x7fa83019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 0x7fa83019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50950/0 (socket says 172.21.15.50:50950) 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 -- 172.21.15.50:0/3228790017 learned_addr learned my addr 172.21.15.50:0/3228790017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa835b3b700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa830109190 0x7fa83019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa834b39700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa830108840 0x7fa83019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 -- 172.21.15.50:0/3228790017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa830109190 msgr2=0x7fa83019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa830109190 0x7fa83019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 -- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa830108840 msgr2=0x7fa83019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa830108840 0x7fa83019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83019f5d0 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.500+0000 7fa83533a700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 0x7fa83019a700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d980 tx=0x7fa82000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820016070 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83019f860 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa820004830 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820005440 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301a71e0 con 0x7fa830103f10 2024-05-17T04:34:02.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.501+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa830063f40 con 0x7fa830103f10 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.503+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa8200055e0 con 0x7fa830103f10 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa82e7fc700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8180b2910 0x7fa8180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa8200e2bf0 con 0x7fa830103f10 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa834b39700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8180b2910 0x7fa8180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa82e7fc700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] conn(0x7fa8180bdbc0 0x7fa8180c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa82001ce50 con 0x7fa8180bdbc0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa8200e3040 con 0x7fa830103f10 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa834b39700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8180b2910 0x7fa8180b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa8240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.504+0000 7fa835b3b700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] conn(0x7fa8180bdbc0 0x7fa8180c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.505+0000 7fa835b3b700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] conn(0x7fa8180bdbc0 0x7fa8180c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.506+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== osd.7 v2:172.21.15.105:6816/2725719637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa82001ce50 con 0x7fa8180bdbc0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.522+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 --> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa830064340 con 0x7fa8180bdbc0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa82e7fc700 1 -- 172.21.15.50:0/3228790017 <== osd.7 v2:172.21.15.105:6816/2725719637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa830064340 con 0x7fa8180bdbc0 2024-05-17T04:34:02.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] conn(0x7fa8180bdbc0 msgr2=0x7fa8180c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] conn(0x7fa8180bdbc0 0x7fa8180c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8180b2910 msgr2=0x7fa8180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa8180b2910 0x7fa8180b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa8240058e0 comp rx=0 tx=0).stop 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 msgr2=0x7fa83019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 --2- 172.21.15.50:0/3228790017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa830103f10 0x7fa83019a700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d980 tx=0x7fa82000dc90 comp rx=0 tx=0).stop 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa835b3b700 1 -- 172.21.15.50:0/3228790017 reap_dead start 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 shutdown_connections 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 >> 172.21.15.50:0/3228790017 conn(0x7fa8300ff9b0 msgr2=0x7fa830101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 shutdown_connections 2024-05-17T04:34:02.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:00.523+0000 7fa83759e700 1 -- 172.21.15.50:0/3228790017 wait complete. 2024-05-17T04:34:02.014 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:01 smithi050 ceph-mon[31739]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-05-17T04:34:02.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:01 smithi105 ceph-mon[34795]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-05-17T04:34:02.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:01 smithi110 ceph-mon[35497]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-05-17T04:34:02.267 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:02.268 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:02.268 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:02.700 INFO:teuthology.orchestra.run.smithi050.stdout:201863462977 2024-05-17T04:34:02.701 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-17T04:34:04.056 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:03 smithi050 ceph-mon[31739]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-17T04:34:04.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:03 smithi110 ceph-mon[35497]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-17T04:34:04.272 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:04.272 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:04.272 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:04.273 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:03 smithi105 ceph-mon[34795]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-05-17T04:34:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:227633266748 2024-05-17T04:34:04.724 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-17T04:34:06.052 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:05 smithi050 ceph-mon[31739]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:34:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:05 smithi105 ceph-mon[34795]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:34:06.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:05 smithi110 ceph-mon[35497]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:34:07.415 INFO:teuthology.orchestra.run.smithi050.stdout:274877906993 2024-05-17T04:34:07.415 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-17T04:34:07.920 INFO:teuthology.orchestra.run.smithi050.stdout:300647710764 2024-05-17T04:34:07.920 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-17T04:34:08.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:07 smithi050 ceph-mon[31739]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:07 smithi105 ceph-mon[34795]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:07 smithi110 ceph-mon[35497]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:09.909 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:09 smithi050 ceph-mon[31739]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-17T04:34:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:09 smithi105 ceph-mon[34795]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-17T04:34:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:09 smithi110 ceph-mon[35497]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-17T04:34:10.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 -- 172.21.15.50:0/3714967099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510103f10 msgr2=0x7fb510104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:10.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3714967099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510103f10 0x7fb510104320 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb500009990 tx=0x7fb500009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 -- 172.21.15.50:0/3714967099 shutdown_connections 2024-05-17T04:34:10.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3714967099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510109190 0x7fb510110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3714967099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510103f10 0x7fb510104320 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3714967099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 0x7fb510108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 -- 172.21.15.50:0/3714967099 >> 172.21.15.50:0/3714967099 conn(0x7fb5100ff9b0 msgr2=0x7fb510101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 -- 172.21.15.50:0/3714967099 shutdown_connections 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.230+0000 7fb5157b4700 1 -- 172.21.15.50:0/3714967099 wait complete. 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 Processor -- start 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 -- start start 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510103f10 0x7fb51019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 0x7fb51019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113c70 con 0x7fb510109190 2024-05-17T04:34:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113af0 con 0x7fb510108840 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb5157b4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb510113df0 con 0x7fb510103f10 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb50f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb50f7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38740/0 (socket says 172.21.15.50:38740) 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb50f7fe700 1 -- 172.21.15.50:0/3257790546 learned_addr learned my addr 172.21.15.50:0/3257790546 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb50e7fc700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 0x7fb51019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.231+0000 7fb50effd700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510103f10 0x7fb51019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 -- 172.21.15.50:0/3257790546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510103f10 msgr2=0x7fb51019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510103f10 0x7fb51019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 -- 172.21.15.50:0/3257790546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 msgr2=0x7fb51019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 0x7fb51019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 -- 172.21.15.50:0/3257790546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51019f490 con 0x7fb510109190 2024-05-17T04:34:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb50f7fe700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb508007d70 tx=0x7fb50800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508016020 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb50800ad40 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51019f780 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080115e0 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.232+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51019fc70 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.233+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb510104320 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.235+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb508019020 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.235+0000 7fb4f7fff700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 0x7fb4f80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.235+0000 7fb50effd700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 0x7fb4f80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.235+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb5080e2550 con 0x7fb510109190 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.236+0000 7fb50effd700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 0x7fb4f80b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb500006010 tx=0x7fb50000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.240+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb508020020 con 0x7fb510109190 2024-05-17T04:34:10.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.454+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb51019b9d0 con 0x7fb510109190 2024-05-17T04:34:10.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.455+0000 7fb4f7fff700 1 -- 172.21.15.50:0/3257790546 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb5080a9e90 con 0x7fb510109190 2024-05-17T04:34:10.456 INFO:teuthology.orchestra.run.smithi050.stdout:154618822733 2024-05-17T04:34:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 msgr2=0x7fb4f80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 0x7fb4f80b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb500006010 tx=0x7fb50000b540 comp rx=0 tx=0).stop 2024-05-17T04:34:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 msgr2=0x7fb51019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb508007d70 tx=0x7fb50800a340 comp rx=0 tx=0).stop 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 shutdown_connections 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4f80b2910 0x7fb4f80b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb510109190 0x7fb51019eea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb510103f10 0x7fb51019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 --2- 172.21.15.50:0/3257790546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb510108840 0x7fb51019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 >> 172.21.15.50:0/3257790546 conn(0x7fb5100ff9b0 msgr2=0x7fb510101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 shutdown_connections 2024-05-17T04:34:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:10.457+0000 7fb5157b4700 1 -- 172.21.15.50:0/3257790546 wait complete. 2024-05-17T04:34:11.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:10 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3257790546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T04:34:11.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:10 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3257790546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T04:34:11.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:10 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3257790546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-17T04:34:11.936 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:11 smithi050 ceph-mon[31739]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 -- 172.21.15.50:0/1040081248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c41091a0 msgr2=0x7f02c4110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 --2- 172.21.15.50:0/1040081248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c41091a0 0x7f02c4110b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00b0a0 tx=0x7f02bc00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 -- 172.21.15.50:0/1040081248 shutdown_connections 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 --2- 172.21.15.50:0/1040081248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c41091a0 0x7f02c4110b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 --2- 172.21.15.50:0/1040081248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4103f20 0x7f02c4104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 --2- 172.21.15.50:0/1040081248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c4108850 0x7f02c4108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 -- 172.21.15.50:0/1040081248 >> 172.21.15.50:0/1040081248 conn(0x7f02c40ff9c0 msgr2=0x7f02c4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 -- 172.21.15.50:0/1040081248 shutdown_connections 2024-05-17T04:34:12.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.053+0000 7f02c9900700 1 -- 172.21.15.50:0/1040081248 wait complete. 2024-05-17T04:34:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 Processor -- start 2024-05-17T04:34:12.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 -- start start 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4108850 0x7f02c41a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c41091a0 0x7f02c419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4113c90 con 0x7f02c4103f20 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4113b10 con 0x7f02c41091a0 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c9900700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02c4113e10 con 0x7f02c4108850 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38770/0 (socket says 172.21.15.50:38770) 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c2ffd700 1 -- 172.21.15.50:0/2035812438 learned_addr learned my addr 172.21.15.50:0/2035812438 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02c37fe700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c41091a0 0x7f02c419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.054+0000 7f02ba7fc700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4108850 0x7f02c41a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:12.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 -- 172.21.15.50:0/2035812438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4108850 msgr2=0x7f02c41a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4108850 0x7f02c41a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 -- 172.21.15.50:0/2035812438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c41091a0 msgr2=0x7f02c419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c41091a0 0x7f02c419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 -- 172.21.15.50:0/2035812438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c419cd80 con 0x7f02c4103f20 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c2ffd700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c920 tx=0x7f02b400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4004a30 con 0x7f02c4103f20 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02b40054b0 con 0x7f02c4103f20 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c419d010 con 0x7f02c4103f20 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4007b70 con 0x7f02c4103f20 2024-05-17T04:34:12.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.055+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c419d550 con 0x7f02c4103f20 2024-05-17T04:34:12.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.056+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c41053d0 con 0x7f02c4103f20 2024-05-17T04:34:12.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.059+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f02b4007d10 con 0x7f02c4103f20 2024-05-17T04:34:12.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.060+0000 7f02c88fe700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 0x7f02b00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:12.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.060+0000 7f02ba7fc700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 0x7f02b00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:12.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.060+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f02b40e2630 con 0x7f02c4103f20 2024-05-17T04:34:12.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.060+0000 7f02ba7fc700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 0x7f02b00b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02c419de80 tx=0x7f02ac00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:12.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.063+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02b4014030 con 0x7f02c4103f20 2024-05-17T04:34:12.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.298+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f02c41a9840 con 0x7f02c4103f20 2024-05-17T04:34:12.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.299+0000 7f02c88fe700 1 -- 172.21.15.50:0/2035812438 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f02b4005660 con 0x7f02c4103f20 2024-05-17T04:34:12.300 INFO:teuthology.orchestra.run.smithi050.stdout:133143986258 2024-05-17T04:34:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 msgr2=0x7f02b00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 0x7f02b00b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02c419de80 tx=0x7f02ac00a380 comp rx=0 tx=0).stop 2024-05-17T04:34:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 msgr2=0x7f02c41a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:12.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c920 tx=0x7f02b400cc30 comp rx=0 tx=0).stop 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 shutdown_connections 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02c41091a0 0x7f02c419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f02b00b2910 0x7f02b00b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f02c4103f20 0x7f02c41a2550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 --2- 172.21.15.50:0/2035812438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02c4108850 0x7f02c41a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 >> 172.21.15.50:0/2035812438 conn(0x7f02c40ff9c0 msgr2=0x7f02c410a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 shutdown_connections 2024-05-17T04:34:12.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:12.301+0000 7f02c9900700 1 -- 172.21.15.50:0/2035812438 wait complete. 2024-05-17T04:34:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:11 smithi105 ceph-mon[34795]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T04:34:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:11 smithi110 ceph-mon[35497]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-17T04:34:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:12 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2035812438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T04:34:13.116 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:12 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2035812438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T04:34:13.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:12 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2035812438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-17T04:34:14.166 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:13 smithi050 ceph-mon[31739]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T04:34:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:13 smithi105 ceph-mon[34795]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T04:34:14.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:13 smithi110 ceph-mon[35497]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-17T04:34:14.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 -- 172.21.15.50:0/2249906065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 msgr2=0x7f314c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 --2- 172.21.15.50:0/2249906065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c103280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3140009990 tx=0x7f3140009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 -- 172.21.15.50:0/2249906065 shutdown_connections 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 --2- 172.21.15.50:0/2249906065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 0x7f314c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 --2- 172.21.15.50:0/2249906065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c103280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 --2- 172.21.15.50:0/2249906065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 0x7f314c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 -- 172.21.15.50:0/2249906065 >> 172.21.15.50:0/2249906065 conn(0x7f314c079ae0 msgr2=0x7f314c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 -- 172.21.15.50:0/2249906065 shutdown_connections 2024-05-17T04:34:14.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.757+0000 7f31542ca700 1 -- 172.21.15.50:0/2249906065 wait complete. 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 Processor -- start 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 -- start start 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 0x7f314c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 0x7f314c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112b80 con 0x7f314c102e70 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112a00 con 0x7f314c107750 2024-05-17T04:34:14.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.758+0000 7f31542ca700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112d00 con 0x7f314c1080a0 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38774/0 (socket says 172.21.15.50:38774) 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 -- 172.21.15.50:0/1902817422 learned_addr learned my addr 172.21.15.50:0/1902817422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3151865700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 0x7f314c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152867700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 0x7f314c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 -- 172.21.15.50:0/1902817422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 msgr2=0x7f314c19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 0x7f314c19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 -- 172.21.15.50:0/1902817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 msgr2=0x7f314c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 0x7f314c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 -- 172.21.15.50:0/1902817422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c19e490 con 0x7f314c102e70 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f3152066700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3140006010 tx=0x7f314000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:14.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140027070 con 0x7f314c102e70 2024-05-17T04:34:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3140022470 con 0x7f314c102e70 2024-05-17T04:34:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.759+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c19e720 con 0x7f314c102e70 2024-05-17T04:34:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.760+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140017650 con 0x7f314c102e70 2024-05-17T04:34:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.760+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c19ec00 con 0x7f314c102e70 2024-05-17T04:34:14.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.761+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c103280 con 0x7f314c102e70 2024-05-17T04:34:14.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.763+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3140009d10 con 0x7f314c102e70 2024-05-17T04:34:14.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.764+0000 7f313f7fe700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 0x7f31380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:14.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.764+0000 7f3151865700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 0x7f31380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:14.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.764+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f31400e3740 con 0x7f314c102e70 2024-05-17T04:34:14.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.765+0000 7f3151865700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 0x7f31380b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f314c1049f0 tx=0x7f314800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:14.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.767+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3140013070 con 0x7f314c102e70 2024-05-17T04:34:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.993+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f314c10aef0 con 0x7f314c102e70 2024-05-17T04:34:14.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.993+0000 7f313f7fe700 1 -- 172.21.15.50:0/1902817422 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31400aafa0 con 0x7f314c102e70 2024-05-17T04:34:14.995 INFO:teuthology.orchestra.run.smithi050.stdout:77309411421 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 msgr2=0x7f31380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 0x7f31380b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f314c1049f0 tx=0x7f314800b410 comp rx=0 tx=0).stop 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 msgr2=0x7f314c1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3140006010 tx=0x7f314000bfc0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 shutdown_connections 2024-05-17T04:34:14.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f314c1080a0 0x7f314c19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f31380b2910 0x7f31380b4e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f314c102e70 0x7f314c1995c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 --2- 172.21.15.50:0/1902817422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f314c107750 0x7f314c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 >> 172.21.15.50:0/1902817422 conn(0x7f314c079ae0 msgr2=0x7f314c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 shutdown_connections 2024-05-17T04:34:14.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:14.997+0000 7f31542ca700 1 -- 172.21.15.50:0/1902817422 wait complete. 2024-05-17T04:34:15.548 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986254 got 133143986258 for osd.4 2024-05-17T04:34:15.549 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:15 smithi050 ceph-mon[31739]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-17T04:34:16.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:15 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1902817422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T04:34:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:15 smithi105 ceph-mon[34795]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-17T04:34:16.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:15 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1902817422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T04:34:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:15 smithi110 ceph-mon[35497]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-17T04:34:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:15 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1902817422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-17T04:34:17.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 -- 172.21.15.50:0/2114004963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4107730 msgr2=0x7f21e4107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/2114004963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4107730 0x7f21e4107b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009990 tx=0x7f21d8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 -- 172.21.15.50:0/2114004963 shutdown_connections 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/2114004963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4108080 0x7f21e410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/2114004963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4102e00 0x7f21e4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/2114004963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4107730 0x7f21e4107b40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 -- 172.21.15.50:0/2114004963 >> 172.21.15.50:0/2114004963 conn(0x7f21e4079bd0 msgr2=0x7f21e4079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 -- 172.21.15.50:0/2114004963 shutdown_connections 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.008+0000 7f21e9d5f700 1 -- 172.21.15.50:0/2114004963 wait complete. 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 Processor -- start 2024-05-17T04:34:17.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 -- start start 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4107730 0x7f21e419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4108080 0x7f21e419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4112b40 con 0x7f21e4102e00 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21e41129c0 con 0x7f21e4108080 2024-05-17T04:34:17.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e9d5f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4112cc0 con 0x7f21e4107730 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38786/0 (socket says 172.21.15.50:38786) 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e37fe700 1 -- 172.21.15.50:0/4128963232 learned_addr learned my addr 172.21.15.50:0/4128963232 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e2ffd700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4107730 0x7f21e419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.009+0000 7f21e3fff700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4108080 0x7f21e419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 -- 172.21.15.50:0/4128963232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4107730 msgr2=0x7f21e419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4107730 0x7f21e419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 -- 172.21.15.50:0/4128963232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4108080 msgr2=0x7f21e419c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:17.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4108080 0x7f21e419c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 -- 172.21.15.50:0/4128963232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e41a69a0 con 0x7f21e4102e00 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e37fe700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f21d400d950 tx=0x7f21d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4016070 con 0x7f21e4102e00 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21d400fd20 con 0x7f21e4102e00 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400e3b0 con 0x7f21e4102e00 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e41a6c90 con 0x7f21e4102e00 2024-05-17T04:34:17.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.010+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41a7150 con 0x7f21e4102e00 2024-05-17T04:34:17.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.011+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e4103210 con 0x7f21e4102e00 2024-05-17T04:34:17.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.013+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f21d4019070 con 0x7f21e4102e00 2024-05-17T04:34:17.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.014+0000 7f21e0ff9700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 0x7f21cc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:17.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.015+0000 7f21e2ffd700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 0x7f21cc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:17.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.015+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f21d40e2160 con 0x7f21e4102e00 2024-05-17T04:34:17.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.015+0000 7f21e2ffd700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 0x7f21cc0b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006010 tx=0x7f21d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:17.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.017+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21d40a9990 con 0x7f21e4102e00 2024-05-17T04:34:17.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.243+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f21e406bc70 con 0x7f21e4102e00 2024-05-17T04:34:17.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.243+0000 7f21e0ff9700 1 -- 172.21.15.50:0/4128963232 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21d40a9290 con 0x7f21e4102e00 2024-05-17T04:34:17.244 INFO:teuthology.orchestra.run.smithi050.stdout:253403070522 2024-05-17T04:34:17.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 msgr2=0x7f21cc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 0x7f21cc0b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006010 tx=0x7f21d8005960 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 msgr2=0x7f21e419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f21d400d950 tx=0x7f21d400dc60 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 shutdown_connections 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e4108080 0x7f21e419c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21cc0b2990 0x7f21cc0b4ea0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21e4102e00 0x7f21e419b6a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 --2- 172.21.15.50:0/4128963232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21e4107730 0x7f21e419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.245+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 >> 172.21.15.50:0/4128963232 conn(0x7f21e4079bd0 msgr2=0x7f21e4105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:17.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.246+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 shutdown_connections 2024-05-17T04:34:17.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:17.246+0000 7f21e9d5f700 1 -- 172.21.15.50:0/4128963232 wait complete. 2024-05-17T04:34:17.583 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822730 got 154618822733 for osd.5 2024-05-17T04:34:17.584 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:18.114 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:17 smithi050 ceph-mon[31739]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:18.114 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:17 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/4128963232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 -- 172.21.15.50:0/2819582945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c107730 msgr2=0x7f6d7c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 --2- 172.21.15.50:0/2819582945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c107730 0x7f6d7c107b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009990 tx=0x7f6d6c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 -- 172.21.15.50:0/2819582945 shutdown_connections 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 --2- 172.21.15.50:0/2819582945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c108080 0x7f6d7c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 --2- 172.21.15.50:0/2819582945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 0x7f6d7c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 --2- 172.21.15.50:0/2819582945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c107730 0x7f6d7c107b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 -- 172.21.15.50:0/2819582945 >> 172.21.15.50:0/2819582945 conn(0x7f6d7c079bd0 msgr2=0x7f6d7c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 -- 172.21.15.50:0/2819582945 shutdown_connections 2024-05-17T04:34:18.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.131+0000 7f6d836a2700 1 -- 172.21.15.50:0/2819582945 wait complete. 2024-05-17T04:34:18.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.132+0000 7f6d836a2700 1 Processor -- start 2024-05-17T04:34:18.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.132+0000 7f6d836a2700 1 -- start start 2024-05-17T04:34:18.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.132+0000 7f6d836a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 0x7f6d7c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.132+0000 7f6d836a2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c107730 0x7f6d7c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d836a2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d836a2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c112b40 con 0x7f6d7c108080 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d836a2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1129c0 con 0x7f6d7c107730 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d836a2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c112cc0 con 0x7f6d7c102e00 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38804/0 (socket says 172.21.15.50:38804) 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 -- 172.21.15.50:0/597571300 learned_addr learned my addr 172.21.15.50:0/597571300 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d8143e700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 0x7f6d7c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d80c3d700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c107730 0x7f6d7c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:18.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 -- 172.21.15.50:0/597571300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 msgr2=0x7f6d7c19b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 0x7f6d7c19b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 -- 172.21.15.50:0/597571300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c107730 msgr2=0x7f6d7c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c107730 0x7f6d7c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 -- 172.21.15.50:0/597571300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c1993a0 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d81c3f700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d74007ee0 tx=0x7f6d7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.133+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74004bb0 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.134+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c199690 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.134+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d74004d50 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.134+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740058b0 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.134+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c199b50 con 0x7f6d7c108080 2024-05-17T04:34:18.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.134+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c103210 con 0x7f6d7c108080 2024-05-17T04:34:18.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.136+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6d7401a020 con 0x7f6d7c108080 2024-05-17T04:34:18.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.137+0000 7f6d727fc700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 0x7f6d680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:18.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.138+0000 7f6d8143e700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 0x7f6d680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:18.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.138+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6d740e2380 con 0x7f6d7c108080 2024-05-17T04:34:18.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.138+0000 7f6d8143e700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 0x7f6d680b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6d78005950 tx=0x7f6d7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:18.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.141+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d74010070 con 0x7f6d7c108080 2024-05-17T04:34:18.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:17 smithi105 ceph-mon[34795]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:18.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:17 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/4128963232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T04:34:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:17 smithi110 ceph-mon[35497]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:17 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/4128963232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-17T04:34:18.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.356+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6d7c19b350 con 0x7f6d7c108080 2024-05-17T04:34:18.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.357+0000 7f6d727fc700 1 -- 172.21.15.50:0/597571300 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d7c19b350 con 0x7f6d7c108080 2024-05-17T04:34:18.358 INFO:teuthology.orchestra.run.smithi050.stdout:55834574947 2024-05-17T04:34:18.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 msgr2=0x7f6d680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:18.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 0x7f6d680b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6d78005950 tx=0x7f6d7800b410 comp rx=0 tx=0).stop 2024-05-17T04:34:18.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 msgr2=0x7f6d7c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:18.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6d74007ee0 tx=0x7f6d7400d3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 shutdown_connections 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6d680b2910 0x7f6d680b4e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d7c108080 0x7f6d7c19bfd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d7c102e00 0x7f6d7c19b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 --2- 172.21.15.50:0/597571300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d7c107730 0x7f6d7c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 >> 172.21.15.50:0/597571300 conn(0x7f6d7c079bd0 msgr2=0x7f6d7c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 shutdown_connections 2024-05-17T04:34:18.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:18.359+0000 7f6d836a2700 1 -- 172.21.15.50:0/597571300 wait complete. 2024-05-17T04:34:19.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:18 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/597571300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T04:34:19.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-05-17T04:34:19.174 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:19.203 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:19.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:18 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/597571300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T04:34:19.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:18 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/597571300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-17T04:34:20.055 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:19 smithi050 ceph-mon[31739]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:20.055 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:19 smithi050 ceph-mon[31739]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3842131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b0104ed0 msgr2=0x7ff5b01052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3842131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b0104ed0 0x7ff5b01052e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009990 tx=0x7ff5a0009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3842131745 shutdown_connections 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3842131745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b010e2b0 0x7ff5b0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3842131745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b0104ed0 0x7ff5b01052e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3842131745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0105820 0x7ff5b010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3842131745 >> 172.21.15.50:0/3842131745 conn(0x7ff5b0079ae0 msgr2=0x7ff5b0079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3842131745 shutdown_connections 2024-05-17T04:34:20.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.173+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3842131745 wait complete. 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.174+0000 7ff5b4de1700 1 Processor -- start 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.174+0000 7ff5b4de1700 1 -- start start 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0104ed0 0x7ff5b019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b0105820 0x7ff5b019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113c70 con 0x7ff5b010e2b0 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113af0 con 0x7ff5b0104ed0 2024-05-17T04:34:20.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5b4de1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0113df0 con 0x7ff5b0105820 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43360/0 (socket says 172.21.15.50:43360) 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5ae59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0104ed0 0x7ff5b019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 -- 172.21.15.50:0/3832896895 learned_addr learned my addr 172.21.15.50:0/3832896895 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5add9b700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b0105820 0x7ff5b019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 -- 172.21.15.50:0/3832896895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b0105820 msgr2=0x7ff5b019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b0105820 0x7ff5b019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 -- 172.21.15.50:0/3832896895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0104ed0 msgr2=0x7ff5b019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0104ed0 0x7ff5b019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 -- 172.21.15.50:0/3832896895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b019f580 con 0x7ff5b010e2b0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.175+0000 7ff5aed9d700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8007ee0 tx=0x7ff5a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.176+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004bb0 con 0x7ff5b010e2b0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.176+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b019f870 con 0x7ff5b010e2b0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.176+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a8004d50 con 0x7ff5b010e2b0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.176+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80058b0 con 0x7ff5b010e2b0 2024-05-17T04:34:20.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.176+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a7190 con 0x7ff5b010e2b0 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.177+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b01052e0 con 0x7ff5b010e2b0 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.178+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff5a801a020 con 0x7ff5b010e2b0 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.179+0000 7ff59f7fe700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 0x7ff5980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.179+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff5a80e2290 con 0x7ff5b010e2b0 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.179+0000 7ff5ae59c700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 0x7ff5980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:20.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.180+0000 7ff5ae59c700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 0x7ff5980b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000b5c0 tx=0x7ff5a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:20.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.182+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5a8010070 con 0x7ff5b010e2b0 2024-05-17T04:34:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:19 smithi105 ceph-mon[34795]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:20.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:19 smithi105 ceph-mon[34795]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T04:34:20.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:19 smithi110 ceph-mon[35497]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:19 smithi110 ceph-mon[35497]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-17T04:34:20.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.400+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff5b002ea80 con 0x7ff5b010e2b0 2024-05-17T04:34:20.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.401+0000 7ff59f7fe700 1 -- 172.21.15.50:0/3832896895 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5a80a9c20 con 0x7ff5b010e2b0 2024-05-17T04:34:20.401 INFO:teuthology.orchestra.run.smithi050.stdout:34359738472 2024-05-17T04:34:20.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 msgr2=0x7ff5980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:20.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 0x7ff5980b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000b5c0 tx=0x7ff5a0005960 comp rx=0 tx=0).stop 2024-05-17T04:34:20.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 msgr2=0x7ff5b019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:20.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8007ee0 tx=0x7ff5a800d3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 shutdown_connections 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5980b2880 0x7ff5980b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5b010e2b0 0x7ff5b019eee0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5b0104ed0 0x7ff5b019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 --2- 172.21.15.50:0/3832896895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5b0105820 0x7ff5b019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 >> 172.21.15.50:0/3832896895 conn(0x7ff5b0079ae0 msgr2=0x7ff5b00ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 shutdown_connections 2024-05-17T04:34:20.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:20.403+0000 7ff5b4de1700 1 -- 172.21.15.50:0/3832896895 wait complete. 2024-05-17T04:34:21.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:20 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3832896895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T04:34:21.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:20 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3832896895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T04:34:21.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:20 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3832896895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-17T04:34:21.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738472 for osd.0 2024-05-17T04:34:21.383 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.743+0000 7fec76540700 1 -- 172.21.15.50:0/1746682903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 msgr2=0x7fec7010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.743+0000 7fec76540700 1 --2- 172.21.15.50:0/1746682903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7010fa00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b0a0 tx=0x7fec6800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 -- 172.21.15.50:0/1746682903 shutdown_connections 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 --2- 172.21.15.50:0/1746682903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7010fa00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 --2- 172.21.15.50:0/1746682903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70102e70 0x7fec70103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 --2- 172.21.15.50:0/1746682903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70107750 0x7fec70107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 -- 172.21.15.50:0/1746682903 >> 172.21.15.50:0/1746682903 conn(0x7fec70079ae0 msgr2=0x7fec70079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 -- 172.21.15.50:0/1746682903 shutdown_connections 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 -- 172.21.15.50:0/1746682903 wait complete. 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 Processor -- start 2024-05-17T04:34:21.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.744+0000 7fec76540700 1 -- start start 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70102e70 0x7fec7019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fec70112b90 con 0x7fec701080a0 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec70112a10 con 0x7fec70102e70 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec76540700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec70112d10 con 0x7fec70107750 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51808/0 (socket says 172.21.15.50:51808) 2024-05-17T04:34:21.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec74add700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 -- 172.21.15.50:0/12752469 learned_addr learned my addr 172.21.15.50:0/12752469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec6ffff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70102e70 0x7fec7019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 -- 172.21.15.50:0/12752469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70102e70 msgr2=0x7fec7019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70102e70 0x7fec7019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 -- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 msgr2=0x7fec7019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec67fff700 1 -- 172.21.15.50:0/12752469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec701a6800 con 0x7fec70107750 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.745+0000 7fec74add700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec67fff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c9e0 tx=0x7fec5800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58007aa0 con 0x7fec70107750 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec701a6a90 con 0x7fec70107750 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec58007c40 con 0x7fec70107750 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58018400 con 0x7fec70107750 2024-05-17T04:34:21.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.746+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a7020 con 0x7fec70107750 2024-05-17T04:34:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.747+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec70103280 con 0x7fec70107750 2024-05-17T04:34:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.749+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec5800f450 con 0x7fec70107750 2024-05-17T04:34:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.750+0000 7fec6dffb700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 0x7fec5c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:21.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.750+0000 7fec6ffff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 0x7fec5c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:21.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.750+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fec580e26b0 con 0x7fec70107750 2024-05-17T04:34:21.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.750+0000 7fec6ffff700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 0x7fec5c0b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec60005950 tx=0x7fec6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:21.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.753+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec58014030 con 0x7fec70107750 2024-05-17T04:34:21.819 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:21.929 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:21 smithi050 ceph-mon[31739]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T04:34:21.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.975+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fec7006c630 con 0x7fec70107750 2024-05-17T04:34:21.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.975+0000 7fec6dffb700 1 -- 172.21.15.50:0/12752469 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec580aa070 con 0x7fec70107750 2024-05-17T04:34:21.975 INFO:teuthology.orchestra.run.smithi050.stdout:107374182490 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 msgr2=0x7fec5c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 0x7fec5c0b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec60005950 tx=0x7fec6000b410 comp rx=0 tx=0).stop 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 msgr2=0x7fec7019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c9e0 tx=0x7fec5800ccf0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 shutdown_connections 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec5c0b2880 0x7fec5c0b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec701080a0 0x7fec7019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec70102e70 0x7fec7019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 --2- 172.21.15.50:0/12752469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec70107750 0x7fec7019ba80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:21.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 >> 172.21.15.50:0/12752469 conn(0x7fec70079ae0 msgr2=0x7fec70105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:21.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 shutdown_connections 2024-05-17T04:34:21.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:21.978+0000 7fec76540700 1 -- 172.21.15.50:0/12752469 wait complete. 2024-05-17T04:34:22.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411421 for osd.2 2024-05-17T04:34:22.315 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:22.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:21 smithi105 ceph-mon[34795]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T04:34:22.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:21 smithi110 ceph-mon[35497]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-17T04:34:22.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070516 got 253403070522 for osd.9 2024-05-17T04:34:22.652 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:23.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:22 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/12752469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T04:34:23.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:22 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/12752469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T04:34:23.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:22 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/12752469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-17T04:34:24.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 -- 172.21.15.50:0/4157796123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac1080a0 msgr2=0x7fe8ac10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/4157796123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac10fa00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009990 tx=0x7fe8a4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 -- 172.21.15.50:0/4157796123 shutdown_connections 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/4157796123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac10fa00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/4157796123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac102e70 0x7fe8ac103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/4157796123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac107750 0x7fe8ac107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 -- 172.21.15.50:0/4157796123 >> 172.21.15.50:0/4157796123 conn(0x7fe8ac079ae0 msgr2=0x7fe8ac079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 -- 172.21.15.50:0/4157796123 shutdown_connections 2024-05-17T04:34:24.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.262+0000 7fe8b1a38700 1 -- 172.21.15.50:0/4157796123 wait complete. 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 Processor -- start 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 -- start start 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac107750 0x7fe8ac19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112b90 con 0x7fe8ac102e70 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112a10 con 0x7fe8ac1080a0 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.263+0000 7fe8b1a38700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac112d10 con 0x7fe8ac107750 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43394/0 (socket says 172.21.15.50:43394) 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 -- 172.21.15.50:0/1780299292 learned_addr learned my addr 172.21.15.50:0/1780299292 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:24.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8ab7fe700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aa7fc700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac107750 0x7fe8ac19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 -- 172.21.15.50:0/1780299292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac107750 msgr2=0x7fe8ac19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac107750 0x7fe8ac19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 -- 172.21.15.50:0/1780299292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac1080a0 msgr2=0x7fe8ac19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 -- 172.21.15.50:0/1780299292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac19f530 con 0x7fe8ac102e70 2024-05-17T04:34:24.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8aaffd700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00ef10 tx=0x7fe89c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:24.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:24.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:34:24.266 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:34:24.267 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.264+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00c9c0 con 0x7fe8ac102e70 2024-05-17T04:34:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.265+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac19f7c0 con 0x7fe8ac102e70 2024-05-17T04:34:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.265+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe89c019070 con 0x7fe8ac102e70 2024-05-17T04:34:24.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.265+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c014480 con 0x7fe8ac102e70 2024-05-17T04:34:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.265+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a7140 con 0x7fe8ac102e70 2024-05-17T04:34:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.266+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac103280 con 0x7fe8ac102e70 2024-05-17T04:34:24.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.268+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe89c012050 con 0x7fe8ac102e70 2024-05-17T04:34:24.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.269+0000 7fe8b0a36700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 0x7fe8980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:24.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.269+0000 7fe8aa7fc700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 0x7fe8980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:24.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.269+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe89c0e2760 con 0x7fe8ac102e70 2024-05-17T04:34:24.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.270+0000 7fe8aa7fc700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 0x7fe8980b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac104980 tx=0x7fe894009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:24.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.272+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe89c010040 con 0x7fe8ac102e70 2024-05-17T04:34:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:34:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:34:24.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:34:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:34:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.493+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe8ac19b9c0 con 0x7fe8ac102e70 2024-05-17T04:34:24.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.493+0000 7fe8b0a36700 1 -- 172.21.15.50:0/1780299292 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe89c0aa0f0 con 0x7fe8ac102e70 2024-05-17T04:34:24.494 INFO:teuthology.orchestra.run.smithi050.stdout:180388626507 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 msgr2=0x7fe8980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 0x7fe8980b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac104980 tx=0x7fe894009260 comp rx=0 tx=0).stop 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 msgr2=0x7fe8ac19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00ef10 tx=0x7fe89c00c3a0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 shutdown_connections 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8ac1080a0 0x7fe8ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe8980b2900 0x7fe8980b4e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8ac102e70 0x7fe8ac19a660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 --2- 172.21.15.50:0/1780299292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8ac107750 0x7fe8ac19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 >> 172.21.15.50:0/1780299292 conn(0x7fe8ac079ae0 msgr2=0x7fe8ac06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 shutdown_connections 2024-05-17T04:34:24.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:24.496+0000 7fe8b1a38700 1 -- 172.21.15.50:0/1780299292 wait complete. 2024-05-17T04:34:25.267 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:25 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1780299292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T04:34:25.302 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:25.306 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:25.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:25 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1780299292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T04:34:25.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:25 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1780299292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-17T04:34:25.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182485 got 107374182490 for osd.3 2024-05-17T04:34:25.782 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:26 smithi105 ceph-mon[34795]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:26.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:26 smithi110 ceph-mon[35497]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:26 smithi050 ceph-mon[31739]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:27.620 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626502 got 180388626507 for osd.6 2024-05-17T04:34:27.620 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.846+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3785041293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94107730 msgr2=0x7f6f94107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.846+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3785041293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94107730 0x7f6f94107b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009990 tx=0x7f6f7c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3785041293 shutdown_connections 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3785041293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 0x7f6f9410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3785041293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94102e00 0x7f6f94103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3785041293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94107730 0x7f6f94107b40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3785041293 >> 172.21.15.50:0/3785041293 conn(0x7f6f94079bd0 msgr2=0x7f6f94079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3785041293 shutdown_connections 2024-05-17T04:34:27.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.847+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3785041293 wait complete. 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 Processor -- start 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 -- start start 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 0x7f6f9419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 0x7f6f9419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:27.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94112b40 con 0x7f6f94102e00 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f941129c0 con 0x7f6f94108080 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f98cc0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94112cc0 con 0x7f6f94107730 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f9259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43408/0 (socket says 172.21.15.50:43408) 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f9259c700 1 -- 172.21.15.50:0/3069898556 learned_addr learned my addr 172.21.15.50:0/3069898556 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f91d9b700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 0x7f6f9419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.848+0000 7f6f92d9d700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 0x7f6f9419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 -- 172.21.15.50:0/3069898556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 msgr2=0x7f6f9419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 0x7f6f9419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 -- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 msgr2=0x7f6f9419b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:27.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 -- 172.21.15.50:0/3069898556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f941a6880 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f9259c700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f91d9b700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 0x7f6f9419ba70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c000c00 tx=0x7f6f7c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c026070 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f941a6b10 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f7c021410 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c017650 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.849+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941a6fc0 con 0x7f6f94107730 2024-05-17T04:34:27.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.850+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f94103210 con 0x7f6f94107730 2024-05-17T04:34:27.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.852+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6f7c029070 con 0x7f6f94107730 2024-05-17T04:34:27.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.853+0000 7f6f8b7fe700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 0x7f6f800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:27.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.853+0000 7f6f9259c700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 0x7f6f800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:27.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.853+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6f7c0e3a00 con 0x7f6f94107730 2024-05-17T04:34:27.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.853+0000 7f6f9259c700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 0x7f6f800b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009c10 tx=0x7f6f8400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:27.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:27.856+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f7c030020 con 0x7f6f94107730 2024-05-17T04:34:28.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.080+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6f9406bc70 con 0x7f6f94107730 2024-05-17T04:34:28.081 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:28 smithi050 ceph-mon[31739]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:28.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.081+0000 7f6f8b7fe700 1 -- 172.21.15.50:0/3069898556 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6f7c013070 con 0x7f6f94107730 2024-05-17T04:34:28.081 INFO:teuthology.orchestra.run.smithi050.stdout:201863462982 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 msgr2=0x7f6f800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 0x7f6f800b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009c10 tx=0x7f6f8400b040 comp rx=0 tx=0).stop 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 msgr2=0x7f6f9419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 0x7f6f9419ba70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c000c00 tx=0x7f6f7c0047b0 comp rx=0 tx=0).stop 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 shutdown_connections 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f94108080 0x7f6f9419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:28.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6f800b2880 0x7f6f800b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:28.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f94102e00 0x7f6f9419b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:28.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 --2- 172.21.15.50:0/3069898556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f94107730 0x7f6f9419ba70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:28.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 >> 172.21.15.50:0/3069898556 conn(0x7f6f94079bd0 msgr2=0x7f6f94105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:28.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 shutdown_connections 2024-05-17T04:34:28.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:28.083+0000 7f6f98cc0700 1 -- 172.21.15.50:0/3069898556 wait complete. 2024-05-17T04:34:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:28 smithi105 ceph-mon[34795]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:28 smithi110 ceph-mon[35497]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:34:29.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:29 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3069898556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T04:34:29.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:29 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3069898556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T04:34:29.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:29 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3069898556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-17T04:34:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:30 smithi105 ceph-mon[34795]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T04:34:30.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:30 smithi110 ceph-mon[35497]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T04:34:30.324 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:30 smithi050 ceph-mon[31739]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-17T04:34:30.675 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462977 got 201863462982 for osd.7 2024-05-17T04:34:30.676 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:32.286 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:32 smithi050 ceph-mon[31739]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:32 smithi105 ceph-mon[34795]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:32 smithi110 ceph-mon[35497]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-17T04:34:32.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 -- 172.21.15.50:0/1966233077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 msgr2=0x7f5d5c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 --2- 172.21.15.50:0/1966233077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c103280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009990 tx=0x7f5d4c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 -- 172.21.15.50:0/1966233077 shutdown_connections 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 --2- 172.21.15.50:0/1966233077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c1080a0 0x7f5d5c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 --2- 172.21.15.50:0/1966233077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c103280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 --2- 172.21.15.50:0/1966233077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c107750 0x7f5d5c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 -- 172.21.15.50:0/1966233077 >> 172.21.15.50:0/1966233077 conn(0x7f5d5c079ae0 msgr2=0x7f5d5c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 -- 172.21.15.50:0/1966233077 shutdown_connections 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.514+0000 7f5d62749700 1 -- 172.21.15.50:0/1966233077 wait complete. 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 Processor -- start 2024-05-17T04:34:32.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 -- start start 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 0x7f5d5c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c1080a0 0x7f5d5c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c113d00 con 0x7f5d5c102e70 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c113b80 con 0x7f5d5c1080a0 2024-05-17T04:34:32.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d62749700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c113e80 con 0x7f5d5c107750 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34096/0 (socket says 172.21.15.50:34096) 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 -- 172.21.15.50:0/2412664914 learned_addr learned my addr 172.21.15.50:0/2412664914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d53fff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 0x7f5d5c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d60ce6700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c1080a0 0x7f5d5c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 -- 172.21.15.50:0/2412664914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 msgr2=0x7f5d5c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 0x7f5d5c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 -- 172.21.15.50:0/2412664914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c1080a0 msgr2=0x7f5d5c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c1080a0 0x7f5d5c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d5bfff700 1 -- 172.21.15.50:0/2412664914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c19f690 con 0x7f5d5c102e70 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.515+0000 7f5d53fff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 0x7f5d5c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d5bfff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c000c00 tx=0x7f5d4c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:32.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c027070 con 0x7f5d5c102e70 2024-05-17T04:34:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d4c00fc70 con 0x7f5d5c102e70 2024-05-17T04:34:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c19f920 con 0x7f5d5c102e70 2024-05-17T04:34:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c022d40 con 0x7f5d5c102e70 2024-05-17T04:34:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.516+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c19fe00 con 0x7f5d5c102e70 2024-05-17T04:34:32.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.517+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c103280 con 0x7f5d5c102e70 2024-05-17T04:34:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.519+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d4c02a070 con 0x7f5d5c102e70 2024-05-17T04:34:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.519+0000 7f5d59ffb700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 0x7f5d480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.520+0000 7f5d53fff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 0x7f5d480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.520+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5d4c0e3740 con 0x7f5d5c102e70 2024-05-17T04:34:32.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.520+0000 7f5d53fff700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 0x7f5d480b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005950 tx=0x7f5d440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:32.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.523+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d4c031080 con 0x7f5d5c102e70 2024-05-17T04:34:32.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.742+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5d5c02ea80 con 0x7f5d5c102e70 2024-05-17T04:34:32.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.743+0000 7f5d59ffb700 1 -- 172.21.15.50:0/2412664914 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5d4c013070 con 0x7f5d5c102e70 2024-05-17T04:34:32.743 INFO:teuthology.orchestra.run.smithi050.stdout:300647710770 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 msgr2=0x7f5d480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 0x7f5d480b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005950 tx=0x7f5d440058e0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 msgr2=0x7f5d5c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c000c00 tx=0x7f5d4c009f90 comp rx=0 tx=0).stop 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 shutdown_connections 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d5c1080a0 0x7f5d5c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5d480b2880 0x7f5d480b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d5c102e70 0x7f5d5c19a7c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 --2- 172.21.15.50:0/2412664914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d5c107750 0x7f5d5c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 >> 172.21.15.50:0/2412664914 conn(0x7f5d5c079ae0 msgr2=0x7f5d5c1055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 shutdown_connections 2024-05-17T04:34:32.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.745+0000 7f5d62749700 1 -- 172.21.15.50:0/2412664914 wait complete. 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 -- 172.21.15.50:0/2619450587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d48105760 msgr2=0x7f1d4810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 --2- 172.21.15.50:0/2619450587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d48105760 0x7f1d4810dcb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009990 tx=0x7f1d34009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 -- 172.21.15.50:0/2619450587 shutdown_connections 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 --2- 172.21.15.50:0/2619450587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d4810e240 0x7f1d48110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 --2- 172.21.15.50:0/2619450587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48104e10 0x7f1d48105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 --2- 172.21.15.50:0/2619450587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d48105760 0x7f1d4810dcb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 -- 172.21.15.50:0/2619450587 >> 172.21.15.50:0/2619450587 conn(0x7f1d48079ae0 msgr2=0x7f1d48079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:32.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 -- 172.21.15.50:0/2619450587 shutdown_connections 2024-05-17T04:34:32.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.748+0000 7f1d4f206700 1 -- 172.21.15.50:0/2619450587 wait complete. 2024-05-17T04:34:32.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 Processor -- start 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 -- start start 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 0x7f1d4819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 0x7f1d4819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113cc0 con 0x7f1d4810e240 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113b40 con 0x7f1d48104e10 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4f206700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d48113e40 con 0x7f1d48105760 2024-05-17T04:34:32.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4d7a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4d7a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34118/0 (socket says 172.21.15.50:34118) 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4d7a3700 1 -- 172.21.15.50:0/923469694 learned_addr learned my addr 172.21.15.50:0/923469694 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d47fff700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 0x7f1d4819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.749+0000 7f1d4cfa2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 0x7f1d4819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 -- 172.21.15.50:0/923469694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 msgr2=0x7f1d4819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 0x7f1d4819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 -- 172.21.15.50:0/923469694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 msgr2=0x7f1d4819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 0x7f1d4819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 -- 172.21.15.50:0/923469694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4819f5e0 con 0x7f1d4810e240 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d47fff700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 0x7f1d4819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4cfa2700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 0x7f1d4819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4d7a3700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007ee0 tx=0x7f1d4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:32.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000dd20 con 0x7f1d4810e240 2024-05-17T04:34:32.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4819f8d0 con 0x7f1d4810e240 2024-05-17T04:34:32.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d40004020 con 0x7f1d4810e240 2024-05-17T04:34:32.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40015660 con 0x7f1d4810e240 2024-05-17T04:34:32.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.750+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4819fe10 con 0x7f1d4810e240 2024-05-17T04:34:32.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.751+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d48105220 con 0x7f1d4810e240 2024-05-17T04:34:32.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.754+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d400041d0 con 0x7f1d4810e240 2024-05-17T04:34:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.755+0000 7f1d45ffb700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 0x7f1d380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.755+0000 7f1d4cfa2700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 0x7f1d380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.755+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1d400e2730 con 0x7f1d4810e240 2024-05-17T04:34:32.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.755+0000 7f1d4cfa2700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 0x7f1d380b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00afd0 tx=0x7f1d3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:32.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.757+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d40010070 con 0x7f1d4810e240 2024-05-17T04:34:32.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.974+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1d4819bb20 con 0x7f1d4810e240 2024-05-17T04:34:32.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.974+0000 7f1d45ffb700 1 -- 172.21.15.50:0/923469694 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1d400aa060 con 0x7f1d4810e240 2024-05-17T04:34:32.974 INFO:teuthology.orchestra.run.smithi050.stdout:274877906999 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 msgr2=0x7f1d380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 0x7f1d380b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00afd0 tx=0x7f1d3c00a380 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 msgr2=0x7f1d4819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d40007ee0 tx=0x7f1d4000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 shutdown_connections 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1d380b2880 0x7f1d380b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1d4810e240 0x7f1d4819eff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d48104e10 0x7f1d4819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 --2- 172.21.15.50:0/923469694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d48105760 0x7f1d4819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.976+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 >> 172.21.15.50:0/923469694 conn(0x7f1d48079ae0 msgr2=0x7f1d480ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.977+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 shutdown_connections 2024-05-17T04:34:32.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:32.977+0000 7f1d4f206700 1 -- 172.21.15.50:0/923469694 wait complete. 2024-05-17T04:34:33.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:33 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2412664914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T04:34:33.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:33 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/923469694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T04:34:33.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:33 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2412664914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T04:34:33.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:33 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/923469694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T04:34:33.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:33 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2412664914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-17T04:34:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:33 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/923469694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-17T04:34:33.816 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906993 got 274877906999 for osd.10 2024-05-17T04:34:33.817 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:34 smithi105 ceph-mon[34795]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-17T04:34:34.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:34 smithi110 ceph-mon[35497]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-17T04:34:34.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:34 smithi050 ceph-mon[31739]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-17T04:34:34.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710764 got 300647710770 for osd.11 2024-05-17T04:34:34.480 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:34.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 -- 172.21.15.50:0/1056322140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08101080a0 msgr2=0x7f081010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:34.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 --2- 172.21.15.50:0/1056322140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08101080a0 0x7f081010fa00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f080800b0a0 tx=0x7f080800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 -- 172.21.15.50:0/1056322140 shutdown_connections 2024-05-17T04:34:34.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 --2- 172.21.15.50:0/1056322140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08101080a0 0x7f081010fa00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 --2- 172.21.15.50:0/1056322140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0810102e70 0x7f0810103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 --2- 172.21.15.50:0/1056322140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810107750 0x7f0810107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 -- 172.21.15.50:0/1056322140 >> 172.21.15.50:0/1056322140 conn(0x7f0810079ae0 msgr2=0x7f0810079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:34.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 -- 172.21.15.50:0/1056322140 shutdown_connections 2024-05-17T04:34:34.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.754+0000 7f0817ee9700 1 -- 172.21.15.50:0/1056322140 wait complete. 2024-05-17T04:34:34.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 Processor -- start 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 -- start start 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810102e70 0x7f0810199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08101080a0 0x7f081019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112b90 con 0x7f0810107750 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112a10 con 0x7f08101080a0 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0817ee9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112d10 con 0x7f0810102e70 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0815484700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0815484700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34140/0 (socket says 172.21.15.50:34140) 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0815484700 1 -- 172.21.15.50:0/913188121 learned_addr learned my addr 172.21.15.50:0/913188121 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0816486700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08101080a0 0x7f081019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:34.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.755+0000 7f0815c85700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810102e70 0x7f0810199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 -- 172.21.15.50:0/913188121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810102e70 msgr2=0x7f0810199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810102e70 0x7f0810199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 -- 172.21.15.50:0/913188121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08101080a0 msgr2=0x7f081019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08101080a0 0x7f081019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 -- 172.21.15.50:0/913188121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081019e440 con 0x7f0810107750 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0815484700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f080000d980 tx=0x7f080000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800017070 con 0x7f0810107750 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f080000fd20 con 0x7f0810107750 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081019e6d0 con 0x7f0810107750 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080000e3b0 con 0x7f0810107750 2024-05-17T04:34:34.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.756+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081006c040 con 0x7f0810107750 2024-05-17T04:34:34.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.757+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08101042d0 con 0x7f0810107750 2024-05-17T04:34:34.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.760+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f080000f890 con 0x7f0810107750 2024-05-17T04:34:34.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.761+0000 7f0806ffd700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 0x7f07fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:34.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.761+0000 7f0815c85700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 0x7f07fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:34.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.761+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f08000e1fe0 con 0x7f0810107750 2024-05-17T04:34:34.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.761+0000 7f0815c85700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 0x7f07fc0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f080c007be0 tx=0x7f080c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:34.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.764+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08000a9930 con 0x7f0810107750 2024-05-17T04:34:34.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.978+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f081006cf50 con 0x7f0810107750 2024-05-17T04:34:34.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.979+0000 7f0806ffd700 1 -- 172.21.15.50:0/913188121 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f08000a9230 con 0x7f0810107750 2024-05-17T04:34:34.979 INFO:teuthology.orchestra.run.smithi050.stdout:227633266755 2024-05-17T04:34:34.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 msgr2=0x7f07fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:34.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 0x7f07fc0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f080c007be0 tx=0x7f080c0073d0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 msgr2=0x7f0810199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:34.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f080000d980 tx=0x7f080000dc90 comp rx=0 tx=0).stop 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 shutdown_connections 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08101080a0 0x7f081019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0810102e70 0x7f0810199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f07fc0b2880 0x7f07fc0b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 --2- 172.21.15.50:0/913188121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0810107750 0x7f0810199b10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 >> 172.21.15.50:0/913188121 conn(0x7f0810079ae0 msgr2=0x7f08101053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 shutdown_connections 2024-05-17T04:34:34.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:34.981+0000 7f0817ee9700 1 -- 172.21.15.50:0/913188121 wait complete. 2024-05-17T04:34:35.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:35 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/913188121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T04:34:35.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:35 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/913188121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T04:34:35.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:35 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/913188121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-17T04:34:35.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266748 got 227633266755 for osd.8 2024-05-17T04:34:35.922 DEBUG:teuthology.parallel:result is None 2024-05-17T04:34:35.922 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-17T04:34:35.922 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T04:34:36.182 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:36.305 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:36 smithi050 ceph-mon[31739]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T04:34:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:36 smithi105 ceph-mon[34795]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T04:34:36.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:36 smithi110 ceph-mon[35497]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 -- 172.21.15.50:0/3211700860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57141082b0 msgr2=0x7f571410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 --2- 172.21.15.50:0/3211700860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57141082b0 0x7f571410fc10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f570c00b0a0 tx=0x7f570c00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 -- 172.21.15.50:0/3211700860 shutdown_connections 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 --2- 172.21.15.50:0/3211700860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57141082b0 0x7f571410fc10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 --2- 172.21.15.50:0/3211700860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5714103550 0x7f5714103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 --2- 172.21.15.50:0/3211700860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 0x7f5714107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 -- 172.21.15.50:0/3211700860 >> 172.21.15.50:0/3211700860 conn(0x7f5714078040 msgr2=0x7f57140784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 -- 172.21.15.50:0/3211700860 shutdown_connections 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 -- 172.21.15.50:0/3211700860 wait complete. 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.143+0000 7f571ab22700 1 Processor -- start 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 -- start start 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 0x7f57141a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57141082b0 0x7f571419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112df0 con 0x7f5714103550 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112c70 con 0x7f57141082b0 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f571ab22700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112f70 con 0x7f5714107960 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34164/0 (socket says 172.21.15.50:34164) 2024-05-17T04:34:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 -- 172.21.15.50:0/3394599067 learned_addr learned my addr 172.21.15.50:0/3394599067 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f5713fff700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 0x7f57141a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57190bf700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57141082b0 0x7f571419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 -- 172.21.15.50:0/3394599067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 msgr2=0x7f57141a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 0x7f57141a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 -- 172.21.15.50:0/3394599067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57141082b0 msgr2=0x7f571419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57141082b0 0x7f571419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.144+0000 7f57188be700 1 -- 172.21.15.50:0/3394599067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419add0 con 0x7f5714103550 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f57188be700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c9b0 tx=0x7f56fc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc007b00 con 0x7f5714103550 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56fc00ce50 con 0x7f5714103550 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419b060 con 0x7f5714103550 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc018660 con 0x7f5714103550 2024-05-17T04:34:38.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.145+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571419b5a0 con 0x7f5714103550 2024-05-17T04:34:38.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.146+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5714104570 con 0x7f5714103550 2024-05-17T04:34:38.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.147+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f56fc00f450 con 0x7f5714103550 2024-05-17T04:34:38.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.148+0000 7f5711ffb700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 0x7f57000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:38.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.148+0000 7f5713fff700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 0x7f57000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:38.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.148+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f56fc0e24d0 con 0x7f5714103550 2024-05-17T04:34:38.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.149+0000 7f5713fff700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 0x7f57000b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f571419bed0 tx=0x7f570400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:38.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.153+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56fc014030 con 0x7f5714103550 2024-05-17T04:34:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:38 smithi105 ceph-mon[34795]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:38 smithi110 ceph-mon[35497]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:38.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:38 smithi050 ceph-mon[31739]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:38.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.364+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5714103960 con 0x7f57000b2880 2024-05-17T04:34:38.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.370+0000 7f5711ffb700 1 -- 172.21.15.50:0/3394599067 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340739 (secure 0 0 0) 0x7f5714103960 con 0x7f57000b2880 2024-05-17T04:34:38.374 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:38.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 msgr2=0x7f57000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:38.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 0x7f57000b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f571419bed0 tx=0x7f570400a380 comp rx=0 tx=0).stop 2024-05-17T04:34:38.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 msgr2=0x7f57141a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:38.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c9b0 tx=0x7f56fc00ccc0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 shutdown_connections 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57141082b0 0x7f571419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f57000b2880 0x7f57000b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5714103550 0x7f57141a05a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 --2- 172.21.15.50:0/3394599067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5714107960 0x7f57141a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 >> 172.21.15.50:0/3394599067 conn(0x7f5714078040 msgr2=0x7f57140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 shutdown_connections 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:38.373+0000 7f571ab22700 1 -- 172.21.15.50:0/3394599067 wait complete. 2024-05-17T04:34:38.378 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-17T04:34:39.135 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-05-17T04:34:36.754991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657597,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":738,"num_read_kb":468,"num_write":303,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377764,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564060,"statfs":{"total":1151940427776,"available":1151553597440,"internally_reserved":0,"allocated":12259328,"data_stored":8128793,"data_compressed":496092,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003419"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710533+0000","last_change":"2024-05-17T04:31:42.946840+0000","last_active":"2024-05-17T04:34:18.710533+0000","last_peered":"2024-05-17T04:34:18.710533+0000","last_clean":"2024-05-17T04:34:18.710533+0000","last_became_active":"2024-05-17T04:31:42.946391+0000","last_became_peered":"2024-05-17T04:31:42.946391+0000","last_unstale":"2024-05-17T04:34:18.710533+0000","last_undegraded":"2024-05-17T04:34:18.710533+0000","last_fullsized":"2024-05-17T04:34:18.710533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:06:29.922596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727177+0000","last_change":"2024-05-17T04:31:42.934805+0000","last_active":"2024-05-17T04:34:18.727177+0000","last_peered":"2024-05-17T04:34:18.727177+0000","last_clean":"2024-05-17T04:34:18.727177+0000","last_became_active":"2024-05-17T04:31:42.933527+0000","last_became_peered":"2024-05-17T04:31:42.933527+0000","last_unstale":"2024-05-17T04:34:18.727177+0000","last_undegraded":"2024-05-17T04:34:18.727177+0000","last_fullsized":"2024-05-17T04:34:18.727177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:31.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728787+0000","last_change":"2024-05-17T04:31:42.935619+0000","last_active":"2024-05-17T04:34:18.728787+0000","last_peered":"2024-05-17T04:34:18.728787+0000","last_clean":"2024-05-17T04:34:18.728787+0000","last_became_active":"2024-05-17T04:31:42.935270+0000","last_became_peered":"2024-05-17T04:31:42.935270+0000","last_unstale":"2024-05-17T04:34:18.728787+0000","last_undegraded":"2024-05-17T04:34:18.728787+0000","last_fullsized":"2024-05-17T04:34:18.728787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:17.059859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710815+0000","last_change":"2024-05-17T04:31:42.928754+0000","last_active":"2024-05-17T04:34:18.710815+0000","last_peered":"2024-05-17T04:34:18.710815+0000","last_clean":"2024-05-17T04:34:18.710815+0000","last_became_active":"2024-05-17T04:31:42.927407+0000","last_became_peered":"2024-05-17T04:31:42.927407+0000","last_unstale":"2024-05-17T04:34:18.710815+0000","last_undegraded":"2024-05-17T04:34:18.710815+0000","last_fullsized":"2024-05-17T04:34:18.710815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:38:33.270224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.695374+0000","last_change":"2024-05-17T04:31:42.929405+0000","last_active":"2024-05-17T04:34:18.695374+0000","last_peered":"2024-05-17T04:34:18.695374+0000","last_clean":"2024-05-17T04:34:18.695374+0000","last_became_active":"2024-05-17T04:31:42.928970+0000","last_became_peered":"2024-05-17T04:31:42.928970+0000","last_unstale":"2024-05-17T04:34:18.695374+0000","last_undegraded":"2024-05-17T04:34:18.695374+0000","last_fullsized":"2024-05-17T04:34:18.695374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:22:39.866681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727284+0000","last_change":"2024-05-17T04:31:42.929773+0000","last_active":"2024-05-17T04:34:18.727284+0000","last_peered":"2024-05-17T04:34:18.727284+0000","last_clean":"2024-05-17T04:34:18.727284+0000","last_became_active":"2024-05-17T04:31:42.929449+0000","last_became_peered":"2024-05-17T04:31:42.929449+0000","last_unstale":"2024-05-17T04:34:18.727284+0000","last_undegraded":"2024-05-17T04:34:18.727284+0000","last_fullsized":"2024-05-17T04:34:18.727284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:05:54.544961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722730+0000","last_change":"2024-05-17T04:31:42.930674+0000","last_active":"2024-05-17T04:34:18.722730+0000","last_peered":"2024-05-17T04:34:18.722730+0000","last_clean":"2024-05-17T04:34:18.722730+0000","last_became_active":"2024-05-17T04:31:42.929752+0000","last_became_peered":"2024-05-17T04:31:42.929752+0000","last_unstale":"2024-05-17T04:34:18.722730+0000","last_undegraded":"2024-05-17T04:34:18.722730+0000","last_fullsized":"2024-05-17T04:34:18.722730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:54:56.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719974+0000","last_change":"2024-05-17T04:31:42.941443+0000","last_active":"2024-05-17T04:34:18.719974+0000","last_peered":"2024-05-17T04:34:18.719974+0000","last_clean":"2024-05-17T04:34:18.719974+0000","last_became_active":"2024-05-17T04:31:42.941052+0000","last_became_peered":"2024-05-17T04:31:42.941052+0000","last_unstale":"2024-05-17T04:34:18.719974+0000","last_undegraded":"2024-05-17T04:34:18.719974+0000","last_fullsized":"2024-05-17T04:34:18.719974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:07.151040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723304+0000","last_change":"2024-05-17T04:31:42.938708+0000","last_active":"2024-05-17T04:34:18.723304+0000","last_peered":"2024-05-17T04:34:18.723304+0000","last_clean":"2024-05-17T04:34:18.723304+0000","last_became_active":"2024-05-17T04:31:42.938330+0000","last_became_peered":"2024-05-17T04:31:42.938330+0000","last_unstale":"2024-05-17T04:34:18.723304+0000","last_undegraded":"2024-05-17T04:34:18.723304+0000","last_fullsized":"2024-05-17T04:34:18.723304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:23.882565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728325+0000","last_change":"2024-05-17T04:31:42.932779+0000","last_active":"2024-05-17T04:34:18.728325+0000","last_peered":"2024-05-17T04:34:18.728325+0000","last_clean":"2024-05-17T04:34:18.728325+0000","last_became_active":"2024-05-17T04:31:42.932429+0000","last_became_peered":"2024-05-17T04:31:42.932429+0000","last_unstale":"2024-05-17T04:34:18.728325+0000","last_undegraded":"2024-05-17T04:34:18.728325+0000","last_fullsized":"2024-05-17T04:34:18.728325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:22:17.139775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724668+0000","last_change":"2024-05-17T04:31:42.932713+0000","last_active":"2024-05-17T04:34:18.724668+0000","last_peered":"2024-05-17T04:34:18.724668+0000","last_clean":"2024-05-17T04:34:18.724668+0000","last_became_active":"2024-05-17T04:31:42.931451+0000","last_became_peered":"2024-05-17T04:31:42.931451+0000","last_unstale":"2024-05-17T04:34:18.724668+0000","last_undegraded":"2024-05-17T04:34:18.724668+0000","last_fullsized":"2024-05-17T04:34:18.724668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:49:03.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720941+0000","last_change":"2024-05-17T04:32:20.057506+0000","last_active":"2024-05-17T04:34:18.720941+0000","last_peered":"2024-05-17T04:34:18.720941+0000","last_clean":"2024-05-17T04:34:18.720941+0000","last_became_active":"2024-05-17T04:32:20.057245+0000","last_became_peered":"2024-05-17T04:32:20.057245+0000","last_unstale":"2024-05-17T04:34:18.720941+0000","last_undegraded":"2024-05-17T04:34:18.720941+0000","last_fullsized":"2024-05-17T04:34:18.720941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:14.714699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727891+0000","last_change":"2024-05-17T04:31:42.930412+0000","last_active":"2024-05-17T04:34:18.727891+0000","last_peered":"2024-05-17T04:34:18.727891+0000","last_clean":"2024-05-17T04:34:18.727891+0000","last_became_active":"2024-05-17T04:31:42.929930+0000","last_became_peered":"2024-05-17T04:31:42.929930+0000","last_unstale":"2024-05-17T04:34:18.727891+0000","last_undegraded":"2024-05-17T04:34:18.727891+0000","last_fullsized":"2024-05-17T04:34:18.727891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:19:12.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727343+0000","last_change":"2024-05-17T04:31:42.932696+0000","last_active":"2024-05-17T04:34:18.727343+0000","last_peered":"2024-05-17T04:34:18.727343+0000","last_clean":"2024-05-17T04:34:18.727343+0000","last_became_active":"2024-05-17T04:31:42.932391+0000","last_became_peered":"2024-05-17T04:31:42.932391+0000","last_unstale":"2024-05-17T04:34:18.727343+0000","last_undegraded":"2024-05-17T04:34:18.727343+0000","last_fullsized":"2024-05-17T04:34:18.727343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:10:13.071352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.712971+0000","last_change":"2024-05-17T04:31:42.943331+0000","last_active":"2024-05-17T04:34:18.712971+0000","last_peered":"2024-05-17T04:34:18.712971+0000","last_clean":"2024-05-17T04:34:18.712971+0000","last_became_active":"2024-05-17T04:31:42.942963+0000","last_became_peered":"2024-05-17T04:31:42.942963+0000","last_unstale":"2024-05-17T04:34:18.712971+0000","last_undegraded":"2024-05-17T04:34:18.712971+0000","last_fullsized":"2024-05-17T04:34:18.712971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:27:33.614843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.726214+0000","last_change":"2024-05-17T04:31:42.935951+0000","last_active":"2024-05-17T04:34:18.726214+0000","last_peered":"2024-05-17T04:34:18.726214+0000","last_clean":"2024-05-17T04:34:18.726214+0000","last_became_active":"2024-05-17T04:31:42.935564+0000","last_became_peered":"2024-05-17T04:31:42.935564+0000","last_unstale":"2024-05-17T04:34:18.726214+0000","last_undegraded":"2024-05-17T04:34:18.726214+0000","last_fullsized":"2024-05-17T04:34:18.726214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:15:28.282780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710892+0000","last_change":"2024-05-17T04:31:42.946858+0000","last_active":"2024-05-17T04:34:18.710892+0000","last_peered":"2024-05-17T04:34:18.710892+0000","last_clean":"2024-05-17T04:34:18.710892+0000","last_became_active":"2024-05-17T04:31:42.946398+0000","last_became_peered":"2024-05-17T04:31:42.946398+0000","last_unstale":"2024-05-17T04:34:18.710892+0000","last_undegraded":"2024-05-17T04:34:18.710892+0000","last_fullsized":"2024-05-17T04:34:18.710892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:51:09.584967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714902+0000","last_change":"2024-05-17T04:31:42.934716+0000","last_active":"2024-05-17T04:34:18.714902+0000","last_peered":"2024-05-17T04:34:18.714902+0000","last_clean":"2024-05-17T04:34:18.714902+0000","last_became_active":"2024-05-17T04:31:42.934379+0000","last_became_peered":"2024-05-17T04:31:42.934379+0000","last_unstale":"2024-05-17T04:34:18.714902+0000","last_undegraded":"2024-05-17T04:34:18.714902+0000","last_fullsized":"2024-05-17T04:34:18.714902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:54:01.154473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713381+0000","last_change":"2024-05-17T04:31:42.926424+0000","last_active":"2024-05-17T04:34:18.713381+0000","last_peered":"2024-05-17T04:34:18.713381+0000","last_clean":"2024-05-17T04:34:18.713381+0000","last_became_active":"2024-05-17T04:31:42.925948+0000","last_became_peered":"2024-05-17T04:31:42.925948+0000","last_unstale":"2024-05-17T04:34:18.713381+0000","last_undegraded":"2024-05-17T04:34:18.713381+0000","last_fullsized":"2024-05-17T04:34:18.713381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:27:48.495395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933287+0000","last_change":"2024-05-17T04:31:55.992551+0000","last_active":"2024-05-17T04:33:20.933287+0000","last_peered":"2024-05-17T04:33:20.933287+0000","last_clean":"2024-05-17T04:33:20.933287+0000","last_became_active":"2024-05-17T04:31:55.992138+0000","last_became_peered":"2024-05-17T04:31:55.992138+0000","last_unstale":"2024-05-17T04:33:20.933287+0000","last_undegraded":"2024-05-17T04:33:20.933287+0000","last_fullsized":"2024-05-17T04:33:20.933287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:43:40.296492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933079+0000","last_change":"2024-05-17T04:31:39.890303+0000","last_active":"2024-05-17T04:33:20.933079+0000","last_peered":"2024-05-17T04:33:20.933079+0000","last_clean":"2024-05-17T04:33:20.933079+0000","last_became_active":"2024-05-17T04:31:39.889970+0000","last_became_peered":"2024-05-17T04:31:39.889970+0000","last_unstale":"2024-05-17T04:33:20.933079+0000","last_undegraded":"2024-05-17T04:33:20.933079+0000","last_fullsized":"2024-05-17T04:33:20.933079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:29:33.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710993+0000","last_change":"2024-05-17T04:31:42.942203+0000","last_active":"2024-05-17T04:34:18.710993+0000","last_peered":"2024-05-17T04:34:18.710993+0000","last_clean":"2024-05-17T04:34:18.710993+0000","last_became_active":"2024-05-17T04:31:42.940563+0000","last_became_peered":"2024-05-17T04:31:42.940563+0000","last_unstale":"2024-05-17T04:34:18.710993+0000","last_undegraded":"2024-05-17T04:34:18.710993+0000","last_fullsized":"2024-05-17T04:34:18.710993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:19:10.813809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934888+0000","last_change":"2024-05-17T04:31:55.988355+0000","last_active":"2024-05-17T04:33:20.934888+0000","last_peered":"2024-05-17T04:33:20.934888+0000","last_clean":"2024-05-17T04:33:20.934888+0000","last_became_active":"2024-05-17T04:31:55.987874+0000","last_became_peered":"2024-05-17T04:31:55.987874+0000","last_unstale":"2024-05-17T04:33:20.934888+0000","last_undegraded":"2024-05-17T04:33:20.934888+0000","last_fullsized":"2024-05-17T04:33:20.934888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:13:30.650959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.101378+0000","last_change":"2024-05-17T04:31:39.896754+0000","last_active":"2024-05-17T04:33:21.101378+0000","last_peered":"2024-05-17T04:33:21.101378+0000","last_clean":"2024-05-17T04:33:21.101378+0000","last_became_active":"2024-05-17T04:31:39.896369+0000","last_became_peered":"2024-05-17T04:31:39.896369+0000","last_unstale":"2024-05-17T04:33:21.101378+0000","last_undegraded":"2024-05-17T04:33:21.101378+0000","last_fullsized":"2024-05-17T04:33:21.101378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:33:03.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728716+0000","last_change":"2024-05-17T04:31:42.938556+0000","last_active":"2024-05-17T04:34:18.728716+0000","last_peered":"2024-05-17T04:34:18.728716+0000","last_clean":"2024-05-17T04:34:18.728716+0000","last_became_active":"2024-05-17T04:31:42.938228+0000","last_became_peered":"2024-05-17T04:31:42.938228+0000","last_unstale":"2024-05-17T04:34:18.728716+0000","last_undegraded":"2024-05-17T04:34:18.728716+0000","last_fullsized":"2024-05-17T04:34:18.728716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:18:32.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935539+0000","last_change":"2024-05-17T04:31:55.991620+0000","last_active":"2024-05-17T04:33:20.935539+0000","last_peered":"2024-05-17T04:33:20.935539+0000","last_clean":"2024-05-17T04:33:20.935539+0000","last_became_active":"2024-05-17T04:31:55.990762+0000","last_became_peered":"2024-05-17T04:31:55.990762+0000","last_unstale":"2024-05-17T04:33:20.935539+0000","last_undegraded":"2024-05-17T04:33:20.935539+0000","last_fullsized":"2024-05-17T04:33:20.935539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:22:49.033835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935911+0000","last_change":"2024-05-17T04:32:20.067654+0000","last_active":"2024-05-17T04:33:20.935911+0000","last_peered":"2024-05-17T04:33:20.935911+0000","last_clean":"2024-05-17T04:33:20.935911+0000","last_became_active":"2024-05-17T04:32:20.062315+0000","last_became_peered":"2024-05-17T04:32:20.062315+0000","last_unstale":"2024-05-17T04:33:20.935911+0000","last_undegraded":"2024-05-17T04:33:20.935911+0000","last_fullsized":"2024-05-17T04:33:20.935911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:32:52.479838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.729123+0000","last_change":"2024-05-17T04:31:42.938076+0000","last_active":"2024-05-17T04:34:18.729123+0000","last_peered":"2024-05-17T04:34:18.729123+0000","last_clean":"2024-05-17T04:34:18.729123+0000","last_became_active":"2024-05-17T04:31:42.937177+0000","last_became_peered":"2024-05-17T04:31:42.937177+0000","last_unstale":"2024-05-17T04:34:18.729123+0000","last_undegraded":"2024-05-17T04:34:18.729123+0000","last_fullsized":"2024-05-17T04:34:18.729123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:08:29.759076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935296+0000","last_change":"2024-05-17T04:31:55.988774+0000","last_active":"2024-05-17T04:33:20.935296+0000","last_peered":"2024-05-17T04:33:20.935296+0000","last_clean":"2024-05-17T04:33:20.935296+0000","last_became_active":"2024-05-17T04:31:55.987934+0000","last_became_peered":"2024-05-17T04:31:55.987934+0000","last_unstale":"2024-05-17T04:33:20.935296+0000","last_undegraded":"2024-05-17T04:33:20.935296+0000","last_fullsized":"2024-05-17T04:33:20.935296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:24:53.408890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935390+0000","last_change":"2024-05-17T04:31:39.896382+0000","last_active":"2024-05-17T04:33:20.935390+0000","last_peered":"2024-05-17T04:33:20.935390+0000","last_clean":"2024-05-17T04:33:20.935390+0000","last_became_active":"2024-05-17T04:31:39.895863+0000","last_became_peered":"2024-05-17T04:31:39.895863+0000","last_unstale":"2024-05-17T04:33:20.935390+0000","last_undegraded":"2024-05-17T04:33:20.935390+0000","last_fullsized":"2024-05-17T04:33:20.935390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:08:37.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725449+0000","last_change":"2024-05-17T04:31:42.938624+0000","last_active":"2024-05-17T04:34:18.725449+0000","last_peered":"2024-05-17T04:34:18.725449+0000","last_clean":"2024-05-17T04:34:18.725449+0000","last_became_active":"2024-05-17T04:31:42.938197+0000","last_became_peered":"2024-05-17T04:31:42.938197+0000","last_unstale":"2024-05-17T04:34:18.725449+0000","last_undegraded":"2024-05-17T04:34:18.725449+0000","last_fullsized":"2024-05-17T04:34:18.725449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:47:14.474259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933704+0000","last_change":"2024-05-17T04:31:55.988254+0000","last_active":"2024-05-17T04:33:20.933704+0000","last_peered":"2024-05-17T04:33:20.933704+0000","last_clean":"2024-05-17T04:33:20.933704+0000","last_became_active":"2024-05-17T04:31:55.987790+0000","last_became_peered":"2024-05-17T04:31:55.987790+0000","last_unstale":"2024-05-17T04:33:20.933704+0000","last_undegraded":"2024-05-17T04:33:20.933704+0000","last_fullsized":"2024-05-17T04:33:20.933704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:07:03.189403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:32.209035+0000","last_change":"2024-05-17T04:31:39.894789+0000","last_active":"2024-05-17T04:34:32.209035+0000","last_peered":"2024-05-17T04:34:32.209035+0000","last_clean":"2024-05-17T04:34:32.209035+0000","last_became_active":"2024-05-17T04:31:39.894404+0000","last_became_peered":"2024-05-17T04:31:39.894404+0000","last_unstale":"2024-05-17T04:34:32.209035+0000","last_undegraded":"2024-05-17T04:34:32.209035+0000","last_fullsized":"2024-05-17T04:34:32.209035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:00:25.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.707239+0000","last_change":"2024-05-17T04:31:42.932430+0000","last_active":"2024-05-17T04:34:18.707239+0000","last_peered":"2024-05-17T04:34:18.707239+0000","last_clean":"2024-05-17T04:34:18.707239+0000","last_became_active":"2024-05-17T04:31:42.932039+0000","last_became_peered":"2024-05-17T04:31:42.932039+0000","last_unstale":"2024-05-17T04:34:18.707239+0000","last_undegraded":"2024-05-17T04:34:18.707239+0000","last_fullsized":"2024-05-17T04:34:18.707239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:46:53.928092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933905+0000","last_change":"2024-05-17T04:31:55.992898+0000","last_active":"2024-05-17T04:33:20.933905+0000","last_peered":"2024-05-17T04:33:20.933905+0000","last_clean":"2024-05-17T04:33:20.933905+0000","last_became_active":"2024-05-17T04:31:55.992459+0000","last_became_peered":"2024-05-17T04:31:55.992459+0000","last_unstale":"2024-05-17T04:33:20.933905+0000","last_undegraded":"2024-05-17T04:33:20.933905+0000","last_fullsized":"2024-05-17T04:33:20.933905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:45:24.888533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.104751+0000","last_change":"2024-05-17T04:31:39.893014+0000","last_active":"2024-05-17T04:33:21.104751+0000","last_peered":"2024-05-17T04:33:21.104751+0000","last_clean":"2024-05-17T04:33:21.104751+0000","last_became_active":"2024-05-17T04:31:39.892602+0000","last_became_peered":"2024-05-17T04:31:39.892602+0000","last_unstale":"2024-05-17T04:33:21.104751+0000","last_undegraded":"2024-05-17T04:33:21.104751+0000","last_fullsized":"2024-05-17T04:33:21.104751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:45:12.742516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722193+0000","last_change":"2024-05-17T04:31:42.945750+0000","last_active":"2024-05-17T04:34:18.722193+0000","last_peered":"2024-05-17T04:34:18.722193+0000","last_clean":"2024-05-17T04:34:18.722193+0000","last_became_active":"2024-05-17T04:31:42.945332+0000","last_became_peered":"2024-05-17T04:31:42.945332+0000","last_unstale":"2024-05-17T04:34:18.722193+0000","last_undegraded":"2024-05-17T04:34:18.722193+0000","last_fullsized":"2024-05-17T04:34:18.722193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:36:00.521489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593707+0000","last_change":"2024-05-17T04:31:55.984513+0000","last_active":"2024-05-17T04:33:21.593707+0000","last_peered":"2024-05-17T04:33:21.593707+0000","last_clean":"2024-05-17T04:33:21.593707+0000","last_became_active":"2024-05-17T04:31:55.984065+0000","last_became_peered":"2024-05-17T04:31:55.984065+0000","last_unstale":"2024-05-17T04:33:21.593707+0000","last_undegraded":"2024-05-17T04:33:21.593707+0000","last_fullsized":"2024-05-17T04:33:21.593707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:07:06.370969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.105961+0000","last_change":"2024-05-17T04:31:39.889551+0000","last_active":"2024-05-17T04:33:21.105961+0000","last_peered":"2024-05-17T04:33:21.105961+0000","last_clean":"2024-05-17T04:33:21.105961+0000","last_became_active":"2024-05-17T04:31:39.889150+0000","last_became_peered":"2024-05-17T04:31:39.889150+0000","last_unstale":"2024-05-17T04:33:21.105961+0000","last_undegraded":"2024-05-17T04:33:21.105961+0000","last_fullsized":"2024-05-17T04:33:21.105961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:14:06.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717119+0000","last_change":"2024-05-17T04:31:42.922236+0000","last_active":"2024-05-17T04:34:18.717119+0000","last_peered":"2024-05-17T04:34:18.717119+0000","last_clean":"2024-05-17T04:34:18.717119+0000","last_became_active":"2024-05-17T04:31:42.921907+0000","last_became_peered":"2024-05-17T04:31:42.921907+0000","last_unstale":"2024-05-17T04:34:18.717119+0000","last_undegraded":"2024-05-17T04:34:18.717119+0000","last_fullsized":"2024-05-17T04:34:18.717119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:10:09.941419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593667+0000","last_change":"2024-05-17T04:31:55.997210+0000","last_active":"2024-05-17T04:33:21.593667+0000","last_peered":"2024-05-17T04:33:21.593667+0000","last_clean":"2024-05-17T04:33:21.593667+0000","last_became_active":"2024-05-17T04:31:55.996795+0000","last_became_peered":"2024-05-17T04:31:55.996795+0000","last_unstale":"2024-05-17T04:33:21.593667+0000","last_undegraded":"2024-05-17T04:33:21.593667+0000","last_fullsized":"2024-05-17T04:33:21.593667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:51:42.271149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107885+0000","last_change":"2024-05-17T04:31:39.891980+0000","last_active":"2024-05-17T04:33:21.107885+0000","last_peered":"2024-05-17T04:33:21.107885+0000","last_clean":"2024-05-17T04:33:21.107885+0000","last_became_active":"2024-05-17T04:31:39.891636+0000","last_became_peered":"2024-05-17T04:31:39.891636+0000","last_unstale":"2024-05-17T04:33:21.107885+0000","last_undegraded":"2024-05-17T04:33:21.107885+0000","last_fullsized":"2024-05-17T04:33:21.107885+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:05:33.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720151+0000","last_change":"2024-05-17T04:31:42.929970+0000","last_active":"2024-05-17T04:34:18.720151+0000","last_peered":"2024-05-17T04:34:18.720151+0000","last_clean":"2024-05-17T04:34:18.720151+0000","last_became_active":"2024-05-17T04:31:42.929587+0000","last_became_peered":"2024-05-17T04:31:42.929587+0000","last_unstale":"2024-05-17T04:34:18.720151+0000","last_undegraded":"2024-05-17T04:34:18.720151+0000","last_fullsized":"2024-05-17T04:34:18.720151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:05:38.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100781+0000","last_change":"2024-05-17T04:31:55.991388+0000","last_active":"2024-05-17T04:33:21.100781+0000","last_peered":"2024-05-17T04:33:21.100781+0000","last_clean":"2024-05-17T04:33:21.100781+0000","last_became_active":"2024-05-17T04:31:55.990979+0000","last_became_peered":"2024-05-17T04:31:55.990979+0000","last_unstale":"2024-05-17T04:33:21.100781+0000","last_undegraded":"2024-05-17T04:33:21.100781+0000","last_fullsized":"2024-05-17T04:33:21.100781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:34.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:33.209876+0000","last_change":"2024-05-17T04:31:39.893076+0000","last_active":"2024-05-17T04:34:33.209876+0000","last_peered":"2024-05-17T04:34:33.209876+0000","last_clean":"2024-05-17T04:34:33.209876+0000","last_became_active":"2024-05-17T04:31:39.892683+0000","last_became_peered":"2024-05-17T04:31:39.892683+0000","last_unstale":"2024-05-17T04:34:33.209876+0000","last_undegraded":"2024-05-17T04:34:33.209876+0000","last_fullsized":"2024-05-17T04:34:33.209876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:32:19.583662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.730271+0000","last_change":"2024-05-17T04:31:42.938431+0000","last_active":"2024-05-17T04:34:18.730271+0000","last_peered":"2024-05-17T04:34:18.730271+0000","last_clean":"2024-05-17T04:34:18.730271+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:34:18.730271+0000","last_undegraded":"2024-05-17T04:34:18.730271+0000","last_fullsized":"2024-05-17T04:34:18.730271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:56:34.505923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934775+0000","last_change":"2024-05-17T04:32:20.065723+0000","last_active":"2024-05-17T04:33:20.934775+0000","last_peered":"2024-05-17T04:33:20.934775+0000","last_clean":"2024-05-17T04:33:20.934775+0000","last_became_active":"2024-05-17T04:32:20.065266+0000","last_became_peered":"2024-05-17T04:32:20.065266+0000","last_unstale":"2024-05-17T04:33:20.934775+0000","last_undegraded":"2024-05-17T04:33:20.934775+0000","last_fullsized":"2024-05-17T04:33:20.934775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:16:54.256206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935134+0000","last_change":"2024-05-17T04:31:39.892324+0000","last_active":"2024-05-17T04:33:20.935134+0000","last_peered":"2024-05-17T04:33:20.935134+0000","last_clean":"2024-05-17T04:33:20.935134+0000","last_became_active":"2024-05-17T04:31:39.891398+0000","last_became_peered":"2024-05-17T04:31:39.891398+0000","last_unstale":"2024-05-17T04:33:20.935134+0000","last_undegraded":"2024-05-17T04:33:20.935134+0000","last_fullsized":"2024-05-17T04:33:20.935134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:06.423894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.721820+0000","last_change":"2024-05-17T04:31:42.948320+0000","last_active":"2024-05-17T04:34:18.721820+0000","last_peered":"2024-05-17T04:34:18.721820+0000","last_clean":"2024-05-17T04:34:18.721820+0000","last_became_active":"2024-05-17T04:31:42.948009+0000","last_became_peered":"2024-05-17T04:31:42.948009+0000","last_unstale":"2024-05-17T04:34:18.721820+0000","last_undegraded":"2024-05-17T04:34:18.721820+0000","last_fullsized":"2024-05-17T04:34:18.721820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:51:28.464925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935832+0000","last_change":"2024-05-17T04:31:55.984602+0000","last_active":"2024-05-17T04:33:20.935832+0000","last_peered":"2024-05-17T04:33:20.935832+0000","last_clean":"2024-05-17T04:33:20.935832+0000","last_became_active":"2024-05-17T04:31:55.984152+0000","last_became_peered":"2024-05-17T04:31:55.984152+0000","last_unstale":"2024-05-17T04:33:20.935832+0000","last_undegraded":"2024-05-17T04:33:20.935832+0000","last_fullsized":"2024-05-17T04:33:20.935832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:31:23.456878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108324+0000","last_change":"2024-05-17T04:31:39.896889+0000","last_active":"2024-05-17T04:33:21.108324+0000","last_peered":"2024-05-17T04:33:21.108324+0000","last_clean":"2024-05-17T04:33:21.108324+0000","last_became_active":"2024-05-17T04:31:39.896516+0000","last_became_peered":"2024-05-17T04:31:39.896516+0000","last_unstale":"2024-05-17T04:33:21.108324+0000","last_undegraded":"2024-05-17T04:33:21.108324+0000","last_fullsized":"2024-05-17T04:33:21.108324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:45:46.214589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724108+0000","last_change":"2024-05-17T04:31:42.928588+0000","last_active":"2024-05-17T04:34:18.724108+0000","last_peered":"2024-05-17T04:34:18.724108+0000","last_clean":"2024-05-17T04:34:18.724108+0000","last_became_active":"2024-05-17T04:31:42.928215+0000","last_became_peered":"2024-05-17T04:31:42.928215+0000","last_unstale":"2024-05-17T04:34:18.724108+0000","last_undegraded":"2024-05-17T04:34:18.724108+0000","last_fullsized":"2024-05-17T04:34:18.724108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:49:31.894516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.104537+0000","last_change":"2024-05-17T04:32:20.063156+0000","last_active":"2024-05-17T04:33:21.104537+0000","last_peered":"2024-05-17T04:33:21.104537+0000","last_clean":"2024-05-17T04:33:21.104537+0000","last_became_active":"2024-05-17T04:32:20.062600+0000","last_became_peered":"2024-05-17T04:32:20.062600+0000","last_unstale":"2024-05-17T04:33:21.104537+0000","last_undegraded":"2024-05-17T04:33:21.104537+0000","last_fullsized":"2024-05-17T04:33:21.104537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:58:04.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:34.210860+0000","last_change":"2024-05-17T04:31:39.896849+0000","last_active":"2024-05-17T04:34:34.210860+0000","last_peered":"2024-05-17T04:34:34.210860+0000","last_clean":"2024-05-17T04:34:34.210860+0000","last_became_active":"2024-05-17T04:31:39.896134+0000","last_became_peered":"2024-05-17T04:31:39.896134+0000","last_unstale":"2024-05-17T04:34:34.210860+0000","last_undegraded":"2024-05-17T04:34:34.210860+0000","last_fullsized":"2024-05-17T04:34:34.210860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:00:27.150148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":171,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.711508+0000","last_change":"2024-05-17T04:31:42.932473+0000","last_active":"2024-05-17T04:34:18.711508+0000","last_peered":"2024-05-17T04:34:18.711508+0000","last_clean":"2024-05-17T04:34:18.711508+0000","last_became_active":"2024-05-17T04:31:42.932120+0000","last_became_peered":"2024-05-17T04:31:42.932120+0000","last_unstale":"2024-05-17T04:34:18.711508+0000","last_undegraded":"2024-05-17T04:34:18.711508+0000","last_fullsized":"2024-05-17T04:34:18.711508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:57:35.056071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108802+0000","last_change":"2024-05-17T04:31:55.999678+0000","last_active":"2024-05-17T04:33:21.108802+0000","last_peered":"2024-05-17T04:33:21.108802+0000","last_clean":"2024-05-17T04:33:21.108802+0000","last_became_active":"2024-05-17T04:31:55.999236+0000","last_became_peered":"2024-05-17T04:31:55.999236+0000","last_unstale":"2024-05-17T04:33:21.108802+0000","last_undegraded":"2024-05-17T04:33:21.108802+0000","last_fullsized":"2024-05-17T04:33:21.108802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:16:27.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933560+0000","last_change":"2024-05-17T04:31:39.894334+0000","last_active":"2024-05-17T04:33:20.933560+0000","last_peered":"2024-05-17T04:33:20.933560+0000","last_clean":"2024-05-17T04:33:20.933560+0000","last_became_active":"2024-05-17T04:31:39.893801+0000","last_became_peered":"2024-05-17T04:31:39.893801+0000","last_unstale":"2024-05-17T04:33:20.933560+0000","last_undegraded":"2024-05-17T04:33:20.933560+0000","last_fullsized":"2024-05-17T04:33:20.933560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:26:07.143110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.712085+0000","last_change":"2024-05-17T04:31:42.934598+0000","last_active":"2024-05-17T04:34:18.712085+0000","last_peered":"2024-05-17T04:34:18.712085+0000","last_clean":"2024-05-17T04:34:18.712085+0000","last_became_active":"2024-05-17T04:31:42.934268+0000","last_became_peered":"2024-05-17T04:31:42.934268+0000","last_unstale":"2024-05-17T04:34:18.712085+0000","last_undegraded":"2024-05-17T04:34:18.712085+0000","last_fullsized":"2024-05-17T04:34:18.712085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:50:13.868834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593396+0000","last_change":"2024-05-17T04:31:55.986888+0000","last_active":"2024-05-17T04:33:21.593396+0000","last_peered":"2024-05-17T04:33:21.593396+0000","last_clean":"2024-05-17T04:33:21.593396+0000","last_became_active":"2024-05-17T04:31:55.986369+0000","last_became_peered":"2024-05-17T04:31:55.986369+0000","last_unstale":"2024-05-17T04:33:21.593396+0000","last_undegraded":"2024-05-17T04:33:21.593396+0000","last_fullsized":"2024-05-17T04:33:21.593396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:42:40.672724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108168+0000","last_change":"2024-05-17T04:31:39.891226+0000","last_active":"2024-05-17T04:33:21.108168+0000","last_peered":"2024-05-17T04:33:21.108168+0000","last_clean":"2024-05-17T04:33:21.108168+0000","last_became_active":"2024-05-17T04:31:39.890376+0000","last_became_peered":"2024-05-17T04:31:39.890376+0000","last_unstale":"2024-05-17T04:33:21.108168+0000","last_undegraded":"2024-05-17T04:33:21.108168+0000","last_fullsized":"2024-05-17T04:33:21.108168+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:13:14.749546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724900+0000","last_change":"2024-05-17T04:32:20.062998+0000","last_active":"2024-05-17T04:34:18.724900+0000","last_peered":"2024-05-17T04:34:18.724900+0000","last_clean":"2024-05-17T04:34:18.724900+0000","last_became_active":"2024-05-17T04:32:20.062485+0000","last_became_peered":"2024-05-17T04:32:20.062485+0000","last_unstale":"2024-05-17T04:34:18.724900+0000","last_undegraded":"2024-05-17T04:34:18.724900+0000","last_fullsized":"2024-05-17T04:34:18.724900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:20:23.822735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.932923+0000","last_change":"2024-05-17T04:31:55.994500+0000","last_active":"2024-05-17T04:33:20.932923+0000","last_peered":"2024-05-17T04:33:20.932923+0000","last_clean":"2024-05-17T04:33:20.932923+0000","last_became_active":"2024-05-17T04:31:55.994130+0000","last_became_peered":"2024-05-17T04:31:55.994130+0000","last_unstale":"2024-05-17T04:33:20.932923+0000","last_undegraded":"2024-05-17T04:33:20.932923+0000","last_fullsized":"2024-05-17T04:33:20.932923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:15:33.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:33.210056+0000","last_change":"2024-05-17T04:31:39.894453+0000","last_active":"2024-05-17T04:34:33.210056+0000","last_peered":"2024-05-17T04:34:33.210056+0000","last_clean":"2024-05-17T04:34:33.210056+0000","last_became_active":"2024-05-17T04:31:39.893628+0000","last_became_peered":"2024-05-17T04:31:39.893628+0000","last_unstale":"2024-05-17T04:34:33.210056+0000","last_undegraded":"2024-05-17T04:34:33.210056+0000","last_fullsized":"2024-05-17T04:34:33.210056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:55:48.098182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722485+0000","last_change":"2024-05-17T04:31:42.939487+0000","last_active":"2024-05-17T04:34:18.722485+0000","last_peered":"2024-05-17T04:34:18.722485+0000","last_clean":"2024-05-17T04:34:18.722485+0000","last_became_active":"2024-05-17T04:31:42.938719+0000","last_became_peered":"2024-05-17T04:31:42.938719+0000","last_unstale":"2024-05-17T04:34:18.722485+0000","last_undegraded":"2024-05-17T04:34:18.722485+0000","last_fullsized":"2024-05-17T04:34:18.722485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:22:42.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106435+0000","last_change":"2024-05-17T04:31:55.998287+0000","last_active":"2024-05-17T04:33:21.106435+0000","last_peered":"2024-05-17T04:33:21.106435+0000","last_clean":"2024-05-17T04:33:21.106435+0000","last_became_active":"2024-05-17T04:31:55.997876+0000","last_became_peered":"2024-05-17T04:31:55.997876+0000","last_unstale":"2024-05-17T04:33:21.106435+0000","last_undegraded":"2024-05-17T04:33:21.106435+0000","last_fullsized":"2024-05-17T04:33:21.106435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:20.523375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934869+0000","last_change":"2024-05-17T04:31:39.896995+0000","last_active":"2024-05-17T04:33:20.934869+0000","last_peered":"2024-05-17T04:33:20.934869+0000","last_clean":"2024-05-17T04:33:20.934869+0000","last_became_active":"2024-05-17T04:31:39.896607+0000","last_became_peered":"2024-05-17T04:31:39.896607+0000","last_unstale":"2024-05-17T04:33:20.934869+0000","last_undegraded":"2024-05-17T04:33:20.934869+0000","last_fullsized":"2024-05-17T04:33:20.934869+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:13:26.761427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714158+0000","last_change":"2024-05-17T04:32:20.061507+0000","last_active":"2024-05-17T04:34:18.714158+0000","last_peered":"2024-05-17T04:34:18.714158+0000","last_clean":"2024-05-17T04:34:18.714158+0000","last_became_active":"2024-05-17T04:32:20.061245+0000","last_became_peered":"2024-05-17T04:32:20.061245+0000","last_unstale":"2024-05-17T04:34:18.714158+0000","last_undegraded":"2024-05-17T04:34:18.714158+0000","last_fullsized":"2024-05-17T04:34:18.714158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:43.800524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107875+0000","last_change":"2024-05-17T04:31:55.991340+0000","last_active":"2024-05-17T04:33:21.107875+0000","last_peered":"2024-05-17T04:33:21.107875+0000","last_clean":"2024-05-17T04:33:21.107875+0000","last_became_active":"2024-05-17T04:31:55.990733+0000","last_became_peered":"2024-05-17T04:31:55.990733+0000","last_unstale":"2024-05-17T04:33:21.107875+0000","last_undegraded":"2024-05-17T04:33:21.107875+0000","last_fullsized":"2024-05-17T04:33:21.107875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:01:47.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593581+0000","last_change":"2024-05-17T04:32:20.070153+0000","last_active":"2024-05-17T04:33:21.593581+0000","last_peered":"2024-05-17T04:33:21.593581+0000","last_clean":"2024-05-17T04:33:21.593581+0000","last_became_active":"2024-05-17T04:32:20.062400+0000","last_became_peered":"2024-05-17T04:32:20.062400+0000","last_unstale":"2024-05-17T04:33:21.593581+0000","last_undegraded":"2024-05-17T04:33:21.593581+0000","last_fullsized":"2024-05-17T04:33:21.593581+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:25:03.332079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714322+0000","last_change":"2024-05-17T04:32:20.064121+0000","last_active":"2024-05-17T04:34:18.714322+0000","last_peered":"2024-05-17T04:34:18.714322+0000","last_clean":"2024-05-17T04:34:18.714322+0000","last_became_active":"2024-05-17T04:32:20.063707+0000","last_became_peered":"2024-05-17T04:32:20.063707+0000","last_unstale":"2024-05-17T04:34:18.714322+0000","last_undegraded":"2024-05-17T04:34:18.714322+0000","last_fullsized":"2024-05-17T04:34:18.714322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:57:45.362980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934905+0000","last_change":"2024-05-17T04:31:56.178977+0000","last_active":"2024-05-17T04:33:20.934905+0000","last_peered":"2024-05-17T04:33:20.934905+0000","last_clean":"2024-05-17T04:33:20.934905+0000","last_became_active":"2024-05-17T04:31:56.178452+0000","last_became_peered":"2024-05-17T04:31:56.178452+0000","last_unstale":"2024-05-17T04:33:20.934905+0000","last_undegraded":"2024-05-17T04:33:20.934905+0000","last_fullsized":"2024-05-17T04:33:20.934905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:14:19.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.932762+0000","last_change":"2024-05-17T04:31:39.897118+0000","last_active":"2024-05-17T04:33:20.932762+0000","last_peered":"2024-05-17T04:33:20.932762+0000","last_clean":"2024-05-17T04:33:20.932762+0000","last_became_active":"2024-05-17T04:31:39.896761+0000","last_became_peered":"2024-05-17T04:31:39.896761+0000","last_unstale":"2024-05-17T04:33:20.932762+0000","last_undegraded":"2024-05-17T04:33:20.932762+0000","last_fullsized":"2024-05-17T04:33:20.932762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:41:50.575131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728124+0000","last_change":"2024-05-17T04:31:42.930510+0000","last_active":"2024-05-17T04:34:18.728124+0000","last_peered":"2024-05-17T04:34:18.728124+0000","last_clean":"2024-05-17T04:34:18.728124+0000","last_became_active":"2024-05-17T04:31:42.930168+0000","last_became_peered":"2024-05-17T04:31:42.930168+0000","last_unstale":"2024-05-17T04:34:18.728124+0000","last_undegraded":"2024-05-17T04:34:18.728124+0000","last_fullsized":"2024-05-17T04:34:18.728124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:18:23.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106222+0000","last_change":"2024-05-17T04:31:55.996783+0000","last_active":"2024-05-17T04:33:21.106222+0000","last_peered":"2024-05-17T04:33:21.106222+0000","last_clean":"2024-05-17T04:33:21.106222+0000","last_became_active":"2024-05-17T04:31:55.996343+0000","last_became_peered":"2024-05-17T04:31:55.996343+0000","last_unstale":"2024-05-17T04:33:21.106222+0000","last_undegraded":"2024-05-17T04:33:21.106222+0000","last_fullsized":"2024-05-17T04:33:21.106222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:41:49.074496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934474+0000","last_change":"2024-05-17T04:31:39.890854+0000","last_active":"2024-05-17T04:33:20.934474+0000","last_peered":"2024-05-17T04:33:20.934474+0000","last_clean":"2024-05-17T04:33:20.934474+0000","last_became_active":"2024-05-17T04:31:39.890379+0000","last_became_peered":"2024-05-17T04:31:39.890379+0000","last_unstale":"2024-05-17T04:33:20.934474+0000","last_undegraded":"2024-05-17T04:33:20.934474+0000","last_fullsized":"2024-05-17T04:33:20.934474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:06.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727953+0000","last_change":"2024-05-17T04:31:42.936992+0000","last_active":"2024-05-17T04:34:18.727953+0000","last_peered":"2024-05-17T04:34:18.727953+0000","last_clean":"2024-05-17T04:34:18.727953+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:34:18.727953+0000","last_undegraded":"2024-05-17T04:34:18.727953+0000","last_fullsized":"2024-05-17T04:34:18.727953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:27:33.067173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106256+0000","last_change":"2024-05-17T04:31:56.178921+0000","last_active":"2024-05-17T04:33:21.106256+0000","last_peered":"2024-05-17T04:33:21.106256+0000","last_clean":"2024-05-17T04:33:21.106256+0000","last_became_active":"2024-05-17T04:31:56.178430+0000","last_became_peered":"2024-05-17T04:31:56.178430+0000","last_unstale":"2024-05-17T04:33:21.106256+0000","last_undegraded":"2024-05-17T04:33:21.106256+0000","last_fullsized":"2024-05-17T04:33:21.106256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:56:37.980197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722951+0000","last_change":"2024-05-17T04:31:42.938586+0000","last_active":"2024-05-17T04:34:18.722951+0000","last_peered":"2024-05-17T04:34:18.722951+0000","last_clean":"2024-05-17T04:34:18.722951+0000","last_became_active":"2024-05-17T04:31:42.937861+0000","last_became_peered":"2024-05-17T04:31:42.937861+0000","last_unstale":"2024-05-17T04:34:18.722951+0000","last_undegraded":"2024-05-17T04:34:18.722951+0000","last_fullsized":"2024-05-17T04:34:18.722951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:03:11.790930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:34.211105+0000","last_change":"2024-05-17T04:31:39.896824+0000","last_active":"2024-05-17T04:34:34.211105+0000","last_peered":"2024-05-17T04:34:34.211105+0000","last_clean":"2024-05-17T04:34:34.211105+0000","last_became_active":"2024-05-17T04:31:39.896463+0000","last_became_peered":"2024-05-17T04:31:39.896463+0000","last_unstale":"2024-05-17T04:34:34.211105+0000","last_undegraded":"2024-05-17T04:34:34.211105+0000","last_fullsized":"2024-05-17T04:34:34.211105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:36:08.351784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934787+0000","last_change":"2024-05-17T04:31:55.993336+0000","last_active":"2024-05-17T04:33:20.934787+0000","last_peered":"2024-05-17T04:33:20.934787+0000","last_clean":"2024-05-17T04:33:20.934787+0000","last_became_active":"2024-05-17T04:31:55.992851+0000","last_became_peered":"2024-05-17T04:31:55.992851+0000","last_unstale":"2024-05-17T04:33:20.934787+0000","last_undegraded":"2024-05-17T04:33:20.934787+0000","last_fullsized":"2024-05-17T04:33:20.934787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:07:47.563166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728581+0000","last_change":"2024-05-17T04:31:42.941862+0000","last_active":"2024-05-17T04:34:18.728581+0000","last_peered":"2024-05-17T04:34:18.728581+0000","last_clean":"2024-05-17T04:34:18.728581+0000","last_became_active":"2024-05-17T04:31:42.940304+0000","last_became_peered":"2024-05-17T04:31:42.940304+0000","last_unstale":"2024-05-17T04:34:18.728581+0000","last_undegraded":"2024-05-17T04:34:18.728581+0000","last_fullsized":"2024-05-17T04:34:18.728581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:46:14.569208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935685+0000","last_change":"2024-05-17T04:31:39.894080+0000","last_active":"2024-05-17T04:33:20.935685+0000","last_peered":"2024-05-17T04:33:20.935685+0000","last_clean":"2024-05-17T04:33:20.935685+0000","last_became_active":"2024-05-17T04:31:39.893646+0000","last_became_peered":"2024-05-17T04:31:39.893646+0000","last_unstale":"2024-05-17T04:33:20.935685+0000","last_undegraded":"2024-05-17T04:33:20.935685+0000","last_fullsized":"2024-05-17T04:33:20.935685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:27:59.844497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934014+0000","last_change":"2024-05-17T04:30:11.017843+0000","last_active":"2024-05-17T04:33:20.934014+0000","last_peered":"2024-05-17T04:33:20.934014+0000","last_clean":"2024-05-17T04:33:20.934014+0000","last_became_active":"2024-05-17T04:30:11.013443+0000","last_became_peered":"2024-05-17T04:30:11.013443+0000","last_unstale":"2024-05-17T04:33:20.934014+0000","last_undegraded":"2024-05-17T04:33:20.934014+0000","last_fullsized":"2024-05-17T04:33:20.934014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_clean_scrub_stamp":"2024-05-17T04:26:43.066168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:27:18.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108420+0000","last_change":"2024-05-17T04:31:56.179071+0000","last_active":"2024-05-17T04:33:21.108420+0000","last_peered":"2024-05-17T04:33:21.108420+0000","last_clean":"2024-05-17T04:33:21.108420+0000","last_became_active":"2024-05-17T04:31:56.178500+0000","last_became_peered":"2024-05-17T04:31:56.178500+0000","last_unstale":"2024-05-17T04:33:21.108420+0000","last_undegraded":"2024-05-17T04:33:21.108420+0000","last_fullsized":"2024-05-17T04:33:21.108420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:48:02.144119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.709620+0000","last_change":"2024-05-17T04:31:42.932232+0000","last_active":"2024-05-17T04:34:18.709620+0000","last_peered":"2024-05-17T04:34:18.709620+0000","last_clean":"2024-05-17T04:34:18.709620+0000","last_became_active":"2024-05-17T04:31:42.931915+0000","last_became_peered":"2024-05-17T04:31:42.931915+0000","last_unstale":"2024-05-17T04:34:18.709620+0000","last_undegraded":"2024-05-17T04:34:18.709620+0000","last_fullsized":"2024-05-17T04:34:18.709620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:06:57.693180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934214+0000","last_change":"2024-05-17T04:31:39.890852+0000","last_active":"2024-05-17T04:33:20.934214+0000","last_peered":"2024-05-17T04:33:20.934214+0000","last_clean":"2024-05-17T04:33:20.934214+0000","last_became_active":"2024-05-17T04:31:39.890417+0000","last_became_peered":"2024-05-17T04:31:39.890417+0000","last_unstale":"2024-05-17T04:33:20.934214+0000","last_undegraded":"2024-05-17T04:33:20.934214+0000","last_fullsized":"2024-05-17T04:33:20.934214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:45:47.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593365+0000","last_change":"2024-05-17T04:31:55.988931+0000","last_active":"2024-05-17T04:33:21.593365+0000","last_peered":"2024-05-17T04:33:21.593365+0000","last_clean":"2024-05-17T04:33:21.593365+0000","last_became_active":"2024-05-17T04:31:55.988366+0000","last_became_peered":"2024-05-17T04:31:55.988366+0000","last_unstale":"2024-05-17T04:33:21.593365+0000","last_undegraded":"2024-05-17T04:33:21.593365+0000","last_fullsized":"2024-05-17T04:33:21.593365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:36:30.684945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593513+0000","last_change":"2024-05-17T04:32:20.059700+0000","last_active":"2024-05-17T04:33:21.593513+0000","last_peered":"2024-05-17T04:33:21.593513+0000","last_clean":"2024-05-17T04:33:21.593513+0000","last_became_active":"2024-05-17T04:32:20.059234+0000","last_became_peered":"2024-05-17T04:32:20.059234+0000","last_unstale":"2024-05-17T04:33:21.593513+0000","last_undegraded":"2024-05-17T04:33:21.593513+0000","last_fullsized":"2024-05-17T04:33:21.593513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:25:47.214832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.706850+0000","last_change":"2024-05-17T04:31:42.926078+0000","last_active":"2024-05-17T04:34:18.706850+0000","last_peered":"2024-05-17T04:34:18.706850+0000","last_clean":"2024-05-17T04:34:18.706850+0000","last_became_active":"2024-05-17T04:31:42.925755+0000","last_became_peered":"2024-05-17T04:31:42.925755+0000","last_unstale":"2024-05-17T04:34:18.706850+0000","last_undegraded":"2024-05-17T04:34:18.706850+0000","last_fullsized":"2024-05-17T04:34:18.706850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:30:16.952810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100687+0000","last_change":"2024-05-17T04:31:56.178973+0000","last_active":"2024-05-17T04:33:21.100687+0000","last_peered":"2024-05-17T04:33:21.100687+0000","last_clean":"2024-05-17T04:33:21.100687+0000","last_became_active":"2024-05-17T04:31:56.178359+0000","last_became_peered":"2024-05-17T04:31:56.178359+0000","last_unstale":"2024-05-17T04:33:21.100687+0000","last_undegraded":"2024-05-17T04:33:21.100687+0000","last_fullsized":"2024-05-17T04:33:21.100687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:22:48.921778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:53.769588+0000","last_change":"2024-05-17T04:31:39.887364+0000","last_active":"2024-05-17T04:33:53.769588+0000","last_peered":"2024-05-17T04:33:53.769588+0000","last_clean":"2024-05-17T04:33:53.769588+0000","last_became_active":"2024-05-17T04:31:39.886961+0000","last_became_peered":"2024-05-17T04:31:39.886961+0000","last_unstale":"2024-05-17T04:33:53.769588+0000","last_undegraded":"2024-05-17T04:33:53.769588+0000","last_fullsized":"2024-05-17T04:33:53.769588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:48:20.417903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722555+0000","last_change":"2024-05-17T04:31:42.933338+0000","last_active":"2024-05-17T04:34:18.722555+0000","last_peered":"2024-05-17T04:34:18.722555+0000","last_clean":"2024-05-17T04:34:18.722555+0000","last_became_active":"2024-05-17T04:31:42.931619+0000","last_became_peered":"2024-05-17T04:31:42.931619+0000","last_unstale":"2024-05-17T04:34:18.722555+0000","last_undegraded":"2024-05-17T04:34:18.722555+0000","last_fullsized":"2024-05-17T04:34:18.722555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:52.565132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933050+0000","last_change":"2024-05-17T04:31:55.990398+0000","last_active":"2024-05-17T04:33:20.933050+0000","last_peered":"2024-05-17T04:33:20.933050+0000","last_clean":"2024-05-17T04:33:20.933050+0000","last_became_active":"2024-05-17T04:31:55.989595+0000","last_became_peered":"2024-05-17T04:31:55.989595+0000","last_unstale":"2024-05-17T04:33:20.933050+0000","last_undegraded":"2024-05-17T04:33:20.933050+0000","last_fullsized":"2024-05-17T04:33:20.933050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:41:37.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.936077+0000","last_change":"2024-05-17T04:31:40.443503+0000","last_active":"2024-05-17T04:33:20.936077+0000","last_peered":"2024-05-17T04:33:20.936077+0000","last_clean":"2024-05-17T04:33:20.936077+0000","last_became_active":"2024-05-17T04:31:40.443094+0000","last_became_peered":"2024-05-17T04:31:40.443094+0000","last_unstale":"2024-05-17T04:33:20.936077+0000","last_undegraded":"2024-05-17T04:33:20.936077+0000","last_fullsized":"2024-05-17T04:33:20.936077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:03:26.837211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723515+0000","last_change":"2024-05-17T04:31:42.933715+0000","last_active":"2024-05-17T04:34:18.723515+0000","last_peered":"2024-05-17T04:34:18.723515+0000","last_clean":"2024-05-17T04:34:18.723515+0000","last_became_active":"2024-05-17T04:31:42.933360+0000","last_became_peered":"2024-05-17T04:31:42.933360+0000","last_unstale":"2024-05-17T04:34:18.723515+0000","last_undegraded":"2024-05-17T04:34:18.723515+0000","last_fullsized":"2024-05-17T04:34:18.723515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:02:30.568910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107267+0000","last_change":"2024-05-17T04:33:21.106627+0000","last_active":"2024-05-17T04:33:21.107267+0000","last_peered":"2024-05-17T04:33:21.107267+0000","last_clean":"2024-05-17T04:33:21.107267+0000","last_became_active":"2024-05-17T04:33:21.106124+0000","last_became_peered":"2024-05-17T04:33:21.106124+0000","last_unstale":"2024-05-17T04:33:21.107267+0000","last_undegraded":"2024-05-17T04:33:21.107267+0000","last_fullsized":"2024-05-17T04:33:21.107267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:12:35.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593206+0000","last_change":"2024-05-17T04:31:39.896901+0000","last_active":"2024-05-17T04:33:21.593206+0000","last_peered":"2024-05-17T04:33:21.593206+0000","last_clean":"2024-05-17T04:33:21.593206+0000","last_became_active":"2024-05-17T04:31:39.896580+0000","last_became_peered":"2024-05-17T04:31:39.896580+0000","last_unstale":"2024-05-17T04:33:21.593206+0000","last_undegraded":"2024-05-17T04:33:21.593206+0000","last_fullsized":"2024-05-17T04:33:21.593206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:26:24.317276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.695913+0000","last_change":"2024-05-17T04:31:42.935437+0000","last_active":"2024-05-17T04:34:18.695913+0000","last_peered":"2024-05-17T04:34:18.695913+0000","last_clean":"2024-05-17T04:34:18.695913+0000","last_became_active":"2024-05-17T04:31:42.935067+0000","last_became_peered":"2024-05-17T04:31:42.935067+0000","last_unstale":"2024-05-17T04:34:18.695913+0000","last_undegraded":"2024-05-17T04:34:18.695913+0000","last_fullsized":"2024-05-17T04:34:18.695913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:56:20.766093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.110306+0000","last_change":"2024-05-17T04:33:21.109697+0000","last_active":"2024-05-17T04:33:21.110306+0000","last_peered":"2024-05-17T04:33:21.110306+0000","last_clean":"2024-05-17T04:33:21.110306+0000","last_became_active":"2024-05-17T04:33:21.109108+0000","last_became_peered":"2024-05-17T04:33:21.109108+0000","last_unstale":"2024-05-17T04:33:21.110306+0000","last_undegraded":"2024-05-17T04:33:21.110306+0000","last_fullsized":"2024-05-17T04:33:21.110306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:21:35.475395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100892+0000","last_change":"2024-05-17T04:31:39.893877+0000","last_active":"2024-05-17T04:33:21.100892+0000","last_peered":"2024-05-17T04:33:21.100892+0000","last_clean":"2024-05-17T04:33:21.100892+0000","last_became_active":"2024-05-17T04:31:39.893479+0000","last_became_peered":"2024-05-17T04:31:39.893479+0000","last_unstale":"2024-05-17T04:33:21.100892+0000","last_undegraded":"2024-05-17T04:33:21.100892+0000","last_fullsized":"2024-05-17T04:33:21.100892+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:46:21.163132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.708977+0000","last_change":"2024-05-17T04:31:42.934449+0000","last_active":"2024-05-17T04:34:18.708977+0000","last_peered":"2024-05-17T04:34:18.708977+0000","last_clean":"2024-05-17T04:34:18.708977+0000","last_became_active":"2024-05-17T04:31:42.934054+0000","last_became_peered":"2024-05-17T04:31:42.934054+0000","last_unstale":"2024-05-17T04:34:18.708977+0000","last_undegraded":"2024-05-17T04:34:18.708977+0000","last_fullsized":"2024-05-17T04:34:18.708977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:35.344333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.936160+0000","last_change":"2024-05-17T04:31:55.986935+0000","last_active":"2024-05-17T04:33:20.936160+0000","last_peered":"2024-05-17T04:33:20.936160+0000","last_clean":"2024-05-17T04:33:20.936160+0000","last_became_active":"2024-05-17T04:31:55.986471+0000","last_became_peered":"2024-05-17T04:31:55.986471+0000","last_unstale":"2024-05-17T04:33:20.936160+0000","last_undegraded":"2024-05-17T04:33:20.936160+0000","last_fullsized":"2024-05-17T04:33:20.936160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:04:09.479135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933187+0000","last_change":"2024-05-17T04:31:39.890000+0000","last_active":"2024-05-17T04:33:20.933187+0000","last_peered":"2024-05-17T04:33:20.933187+0000","last_clean":"2024-05-17T04:33:20.933187+0000","last_became_active":"2024-05-17T04:31:39.887979+0000","last_became_peered":"2024-05-17T04:31:39.887979+0000","last_unstale":"2024-05-17T04:33:20.933187+0000","last_undegraded":"2024-05-17T04:33:20.933187+0000","last_fullsized":"2024-05-17T04:33:20.933187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:44:13.580387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719173+0000","last_change":"2024-05-17T04:31:42.943980+0000","last_active":"2024-05-17T04:34:18.719173+0000","last_peered":"2024-05-17T04:34:18.719173+0000","last_clean":"2024-05-17T04:34:18.719173+0000","last_became_active":"2024-05-17T04:31:42.943647+0000","last_became_peered":"2024-05-17T04:31:42.943647+0000","last_unstale":"2024-05-17T04:34:18.719173+0000","last_undegraded":"2024-05-17T04:34:18.719173+0000","last_fullsized":"2024-05-17T04:34:18.719173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:17:15.909198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935511+0000","last_change":"2024-05-17T04:31:56.178896+0000","last_active":"2024-05-17T04:33:20.935511+0000","last_peered":"2024-05-17T04:33:20.935511+0000","last_clean":"2024-05-17T04:33:20.935511+0000","last_became_active":"2024-05-17T04:31:56.178298+0000","last_became_peered":"2024-05-17T04:31:56.178298+0000","last_unstale":"2024-05-17T04:33:20.935511+0000","last_undegraded":"2024-05-17T04:33:20.935511+0000","last_fullsized":"2024-05-17T04:33:20.935511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:30:32.759024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935193+0000","last_change":"2024-05-17T04:31:39.894705+0000","last_active":"2024-05-17T04:33:20.935193+0000","last_peered":"2024-05-17T04:33:20.935193+0000","last_clean":"2024-05-17T04:33:20.935193+0000","last_became_active":"2024-05-17T04:31:39.894312+0000","last_became_peered":"2024-05-17T04:31:39.894312+0000","last_unstale":"2024-05-17T04:33:20.935193+0000","last_undegraded":"2024-05-17T04:33:20.935193+0000","last_fullsized":"2024-05-17T04:33:20.935193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:41:13.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725949+0000","last_change":"2024-05-17T04:31:42.941171+0000","last_active":"2024-05-17T04:34:18.725949+0000","last_peered":"2024-05-17T04:34:18.725949+0000","last_clean":"2024-05-17T04:34:18.725949+0000","last_became_active":"2024-05-17T04:31:42.940724+0000","last_became_peered":"2024-05-17T04:31:42.940724+0000","last_unstale":"2024-05-17T04:34:18.725949+0000","last_undegraded":"2024-05-17T04:34:18.725949+0000","last_fullsized":"2024-05-17T04:34:18.725949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:26:08.933000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593215+0000","last_change":"2024-05-17T04:31:55.992427+0000","last_active":"2024-05-17T04:33:21.593215+0000","last_peered":"2024-05-17T04:33:21.593215+0000","last_clean":"2024-05-17T04:33:21.593215+0000","last_became_active":"2024-05-17T04:31:55.991936+0000","last_became_peered":"2024-05-17T04:31:55.991936+0000","last_unstale":"2024-05-17T04:33:21.593215+0000","last_undegraded":"2024-05-17T04:33:21.593215+0000","last_fullsized":"2024-05-17T04:33:21.593215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:30:01.610347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593298+0000","last_change":"2024-05-17T04:32:20.062349+0000","last_active":"2024-05-17T04:33:21.593298+0000","last_peered":"2024-05-17T04:33:21.593298+0000","last_clean":"2024-05-17T04:33:21.593298+0000","last_became_active":"2024-05-17T04:32:20.061708+0000","last_became_peered":"2024-05-17T04:32:20.061708+0000","last_unstale":"2024-05-17T04:33:21.593298+0000","last_undegraded":"2024-05-17T04:33:21.593298+0000","last_fullsized":"2024-05-17T04:33:21.593298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:51:02.946670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725338+0000","last_change":"2024-05-17T04:31:42.931998+0000","last_active":"2024-05-17T04:34:18.725338+0000","last_peered":"2024-05-17T04:34:18.725338+0000","last_clean":"2024-05-17T04:34:18.725338+0000","last_became_active":"2024-05-17T04:31:42.929898+0000","last_became_peered":"2024-05-17T04:31:42.929898+0000","last_unstale":"2024-05-17T04:34:18.725338+0000","last_undegraded":"2024-05-17T04:34:18.725338+0000","last_fullsized":"2024-05-17T04:34:18.725338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:55:35.349637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593250+0000","last_change":"2024-05-17T04:31:55.990093+0000","last_active":"2024-05-17T04:33:21.593250+0000","last_peered":"2024-05-17T04:33:21.593250+0000","last_clean":"2024-05-17T04:33:21.593250+0000","last_became_active":"2024-05-17T04:31:55.989665+0000","last_became_peered":"2024-05-17T04:31:55.989665+0000","last_unstale":"2024-05-17T04:33:21.593250+0000","last_undegraded":"2024-05-17T04:33:21.593250+0000","last_fullsized":"2024-05-17T04:33:21.593250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:37:43.727322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108607+0000","last_change":"2024-05-17T04:31:39.893601+0000","last_active":"2024-05-17T04:33:21.108607+0000","last_peered":"2024-05-17T04:33:21.108607+0000","last_clean":"2024-05-17T04:33:21.108607+0000","last_became_active":"2024-05-17T04:31:39.893141+0000","last_became_peered":"2024-05-17T04:31:39.893141+0000","last_unstale":"2024-05-17T04:33:21.108607+0000","last_undegraded":"2024-05-17T04:33:21.108607+0000","last_fullsized":"2024-05-17T04:33:21.108607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:05:09.173607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723513+0000","last_change":"2024-05-17T04:31:42.932083+0000","last_active":"2024-05-17T04:34:18.723513+0000","last_peered":"2024-05-17T04:34:18.723513+0000","last_clean":"2024-05-17T04:34:18.723513+0000","last_became_active":"2024-05-17T04:31:42.931520+0000","last_became_peered":"2024-05-17T04:31:42.931520+0000","last_unstale":"2024-05-17T04:34:18.723513+0000","last_undegraded":"2024-05-17T04:34:18.723513+0000","last_fullsized":"2024-05-17T04:34:18.723513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:47:55.057982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107506+0000","last_change":"2024-05-17T04:31:55.990315+0000","last_active":"2024-05-17T04:33:21.107506+0000","last_peered":"2024-05-17T04:33:21.107506+0000","last_clean":"2024-05-17T04:33:21.107506+0000","last_became_active":"2024-05-17T04:31:55.989565+0000","last_became_peered":"2024-05-17T04:31:55.989565+0000","last_unstale":"2024-05-17T04:33:21.107506+0000","last_undegraded":"2024-05-17T04:33:21.107506+0000","last_fullsized":"2024-05-17T04:33:21.107506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:03.699211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106305+0000","last_change":"2024-05-17T04:31:39.890621+0000","last_active":"2024-05-17T04:33:21.106305+0000","last_peered":"2024-05-17T04:33:21.106305+0000","last_clean":"2024-05-17T04:33:21.106305+0000","last_became_active":"2024-05-17T04:31:39.890180+0000","last_became_peered":"2024-05-17T04:31:39.890180+0000","last_unstale":"2024-05-17T04:33:21.106305+0000","last_undegraded":"2024-05-17T04:33:21.106305+0000","last_fullsized":"2024-05-17T04:33:21.106305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:42.018482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.718579+0000","last_change":"2024-05-17T04:31:42.928951+0000","last_active":"2024-05-17T04:34:18.718579+0000","last_peered":"2024-05-17T04:34:18.718579+0000","last_clean":"2024-05-17T04:34:18.718579+0000","last_became_active":"2024-05-17T04:31:42.928382+0000","last_became_peered":"2024-05-17T04:31:42.928382+0000","last_unstale":"2024-05-17T04:34:18.718579+0000","last_undegraded":"2024-05-17T04:34:18.718579+0000","last_fullsized":"2024-05-17T04:34:18.718579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:36.307351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714846+0000","last_change":"2024-05-17T04:31:42.926377+0000","last_active":"2024-05-17T04:34:18.714846+0000","last_peered":"2024-05-17T04:34:18.714846+0000","last_clean":"2024-05-17T04:34:18.714846+0000","last_became_active":"2024-05-17T04:31:42.926007+0000","last_became_peered":"2024-05-17T04:31:42.926007+0000","last_unstale":"2024-05-17T04:34:18.714846+0000","last_undegraded":"2024-05-17T04:34:18.714846+0000","last_fullsized":"2024-05-17T04:34:18.714846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:43:45.456144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719776+0000","last_change":"2024-05-17T04:31:42.932742+0000","last_active":"2024-05-17T04:34:18.719776+0000","last_peered":"2024-05-17T04:34:18.719776+0000","last_clean":"2024-05-17T04:34:18.719776+0000","last_became_active":"2024-05-17T04:31:42.932374+0000","last_became_peered":"2024-05-17T04:31:42.932374+0000","last_unstale":"2024-05-17T04:34:18.719776+0000","last_undegraded":"2024-05-17T04:34:18.719776+0000","last_fullsized":"2024-05-17T04:34:18.719776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:37.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713029+0000","last_change":"2024-05-17T04:31:42.940870+0000","last_active":"2024-05-17T04:34:18.713029+0000","last_peered":"2024-05-17T04:34:18.713029+0000","last_clean":"2024-05-17T04:34:18.713029+0000","last_became_active":"2024-05-17T04:31:42.940463+0000","last_became_peered":"2024-05-17T04:31:42.940463+0000","last_unstale":"2024-05-17T04:34:18.713029+0000","last_undegraded":"2024-05-17T04:34:18.713029+0000","last_fullsized":"2024-05-17T04:34:18.713029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:48:52.684601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720264+0000","last_change":"2024-05-17T04:31:42.936055+0000","last_active":"2024-05-17T04:34:18.720264+0000","last_peered":"2024-05-17T04:34:18.720264+0000","last_clean":"2024-05-17T04:34:18.720264+0000","last_became_active":"2024-05-17T04:31:42.934013+0000","last_became_peered":"2024-05-17T04:31:42.934013+0000","last_unstale":"2024-05-17T04:34:18.720264+0000","last_undegraded":"2024-05-17T04:34:18.720264+0000","last_fullsized":"2024-05-17T04:34:18.720264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:03:10.502727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.709557+0000","last_change":"2024-05-17T04:31:42.942930+0000","last_active":"2024-05-17T04:34:18.709557+0000","last_peered":"2024-05-17T04:34:18.709557+0000","last_clean":"2024-05-17T04:34:18.709557+0000","last_became_active":"2024-05-17T04:31:42.942585+0000","last_became_peered":"2024-05-17T04:31:42.942585+0000","last_unstale":"2024-05-17T04:34:18.709557+0000","last_undegraded":"2024-05-17T04:34:18.709557+0000","last_fullsized":"2024-05-17T04:34:18.709557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:28:45.780505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714651+0000","last_change":"2024-05-17T04:31:42.923752+0000","last_active":"2024-05-17T04:34:18.714651+0000","last_peered":"2024-05-17T04:34:18.714651+0000","last_clean":"2024-05-17T04:34:18.714651+0000","last_became_active":"2024-05-17T04:31:42.923375+0000","last_became_peered":"2024-05-17T04:31:42.923375+0000","last_unstale":"2024-05-17T04:34:18.714651+0000","last_undegraded":"2024-05-17T04:34:18.714651+0000","last_fullsized":"2024-05-17T04:34:18.714651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:32:13.755314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.731053+0000","last_change":"2024-05-17T04:31:42.928666+0000","last_active":"2024-05-17T04:34:18.731053+0000","last_peered":"2024-05-17T04:34:18.731053+0000","last_clean":"2024-05-17T04:34:18.731053+0000","last_became_active":"2024-05-17T04:31:42.928291+0000","last_became_peered":"2024-05-17T04:31:42.928291+0000","last_unstale":"2024-05-17T04:34:18.731053+0000","last_undegraded":"2024-05-17T04:34:18.731053+0000","last_fullsized":"2024-05-17T04:34:18.731053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:12:32.253374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.707560+0000","last_change":"2024-05-17T04:31:42.926316+0000","last_active":"2024-05-17T04:34:18.707560+0000","last_peered":"2024-05-17T04:34:18.707560+0000","last_clean":"2024-05-17T04:34:18.707560+0000","last_became_active":"2024-05-17T04:31:42.925823+0000","last_became_peered":"2024-05-17T04:31:42.925823+0000","last_unstale":"2024-05-17T04:34:18.707560+0000","last_undegraded":"2024-05-17T04:34:18.707560+0000","last_fullsized":"2024-05-17T04:34:18.707560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:16:15.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717555+0000","last_change":"2024-05-17T04:31:42.945587+0000","last_active":"2024-05-17T04:34:18.717555+0000","last_peered":"2024-05-17T04:34:18.717555+0000","last_clean":"2024-05-17T04:34:18.717555+0000","last_became_active":"2024-05-17T04:31:42.945238+0000","last_became_peered":"2024-05-17T04:31:42.945238+0000","last_unstale":"2024-05-17T04:34:18.717555+0000","last_undegraded":"2024-05-17T04:34:18.717555+0000","last_fullsized":"2024-05-17T04:34:18.717555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:17:29.584023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717028+0000","last_change":"2024-05-17T04:31:42.925883+0000","last_active":"2024-05-17T04:34:18.717028+0000","last_peered":"2024-05-17T04:34:18.717028+0000","last_clean":"2024-05-17T04:34:18.717028+0000","last_became_active":"2024-05-17T04:31:42.925506+0000","last_became_peered":"2024-05-17T04:31:42.925506+0000","last_unstale":"2024-05-17T04:34:18.717028+0000","last_undegraded":"2024-05-17T04:34:18.717028+0000","last_fullsized":"2024-05-17T04:34:18.717028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:31:16.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.730488+0000","last_change":"2024-05-17T04:31:42.928444+0000","last_active":"2024-05-17T04:34:18.730488+0000","last_peered":"2024-05-17T04:34:18.730488+0000","last_clean":"2024-05-17T04:34:18.730488+0000","last_became_active":"2024-05-17T04:31:42.928060+0000","last_became_peered":"2024-05-17T04:31:42.928060+0000","last_unstale":"2024-05-17T04:34:18.730488+0000","last_undegraded":"2024-05-17T04:34:18.730488+0000","last_fullsized":"2024-05-17T04:34:18.730488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:47:29.714401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.711617+0000","last_change":"2024-05-17T04:31:42.937010+0000","last_active":"2024-05-17T04:34:18.711617+0000","last_peered":"2024-05-17T04:34:18.711617+0000","last_clean":"2024-05-17T04:34:18.711617+0000","last_became_active":"2024-05-17T04:31:42.936675+0000","last_became_peered":"2024-05-17T04:31:42.936675+0000","last_unstale":"2024-05-17T04:34:18.711617+0000","last_undegraded":"2024-05-17T04:34:18.711617+0000","last_fullsized":"2024-05-17T04:34:18.711617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:06:22.421966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713643+0000","last_change":"2024-05-17T04:31:42.924943+0000","last_active":"2024-05-17T04:34:18.713643+0000","last_peered":"2024-05-17T04:34:18.713643+0000","last_clean":"2024-05-17T04:34:18.713643+0000","last_became_active":"2024-05-17T04:31:42.924578+0000","last_became_peered":"2024-05-17T04:31:42.924578+0000","last_unstale":"2024-05-17T04:34:18.713643+0000","last_undegraded":"2024-05-17T04:34:18.713643+0000","last_fullsized":"2024-05-17T04:34:18.713643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:22:58.614038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284318,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66995,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":662,"num_read_kb":404,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":515050,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1165127,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":519338,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":515118,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1111744,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510418,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":573231,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":511806,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":573949,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510612,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":515600,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822739,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1106800,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T04:34:39.140 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:40.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:40 smithi050 ceph-mon[31739]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:40.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:40 smithi050 ceph-mon[31739]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:40 smithi105 ceph-mon[34795]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:40.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:40 smithi105 ceph-mon[34795]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:40 smithi110 ceph-mon[35497]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:40 smithi110 ceph-mon[35497]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:41.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 -- 172.21.15.50:0/62904022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814103f20 msgr2=0x7f9814104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:41.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 --2- 172.21.15.50:0/62904022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814103f20 0x7f9814104330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f980400b0a0 tx=0x7f980400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 -- 172.21.15.50:0/62904022 shutdown_connections 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 --2- 172.21.15.50:0/62904022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98141091a0 0x7f9814110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 --2- 172.21.15.50:0/62904022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814103f20 0x7f9814104330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 --2- 172.21.15.50:0/62904022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9814108850 0x7f9814108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 -- 172.21.15.50:0/62904022 >> 172.21.15.50:0/62904022 conn(0x7f98140ff9c0 msgr2=0x7f9814101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 -- 172.21.15.50:0/62904022 shutdown_connections 2024-05-17T04:34:41.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.309+0000 7f98191a9700 1 -- 172.21.15.50:0/62904022 wait complete. 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 Processor -- start 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 -- start start 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 0x7f98141a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 0x7f981419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9814113c80 con 0x7f9814108850 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9814113b00 con 0x7f98141091a0 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f98191a9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9814113e00 con 0x7f9814103f20 2024-05-17T04:34:41.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f981259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f981259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40662/0 (socket says 172.21.15.50:40662) 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.310+0000 7f981259c700 1 -- 172.21.15.50:0/118257924 learned_addr learned my addr 172.21.15.50:0/118257924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f9812d9d700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 0x7f98141a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981359e700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 0x7f981419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 -- 172.21.15.50:0/118257924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 msgr2=0x7f98141a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 0x7f98141a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 -- 172.21.15.50:0/118257924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 msgr2=0x7f981419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 0x7f981419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 -- 172.21.15.50:0/118257924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981419ad70 con 0x7f9814108850 2024-05-17T04:34:41.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981359e700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 0x7f981419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f9812d9d700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 0x7f98141a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f981259c700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00e970 tx=0x7f97fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00cbd0 con 0x7f9814108850 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97fc00cd70 con 0x7f9814108850 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981419b060 con 0x7f9814108850 2024-05-17T04:34:41.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc005610 con 0x7f9814108850 2024-05-17T04:34:41.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.311+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141a7290 con 0x7f9814108850 2024-05-17T04:34:41.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.312+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9814104330 con 0x7f9814108850 2024-05-17T04:34:41.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.314+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f97fc004750 con 0x7f9814108850 2024-05-17T04:34:41.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.315+0000 7f980bfff700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:41.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.315+0000 7f9812d9d700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:41.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.315+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97fc0e2b10 con 0x7f9814108850 2024-05-17T04:34:41.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.316+0000 7f9812d9d700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 0x7f98000b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98040062a0 tx=0x7f9804006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:41.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.319+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97fc054df0 con 0x7f9814108850 2024-05-17T04:34:41.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.527+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f981402ea80 con 0x7f98000b2880 2024-05-17T04:34:41.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.533+0000 7f980bfff700 1 -- 172.21.15.50:0/118257924 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340739 (secure 0 0 0) 0x7f981402ea80 con 0x7f98000b2880 2024-05-17T04:34:41.537 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 msgr2=0x7f98000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 0x7f98000b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98040062a0 tx=0x7f9804006210 comp rx=0 tx=0).stop 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 msgr2=0x7f98141a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00e970 tx=0x7f97fc00ed30 comp rx=0 tx=0).stop 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 shutdown_connections 2024-05-17T04:34:41.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98141091a0 0x7f981419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9814103f20 0x7f98141a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f98000b2880 0x7f98000b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 --2- 172.21.15.50:0/118257924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9814108850 0x7f98141a09d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 >> 172.21.15.50:0/118257924 conn(0x7f98140ff9c0 msgr2=0x7f9814101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 shutdown_connections 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:41.537+0000 7f98191a9700 1 -- 172.21.15.50:0/118257924 wait complete. 2024-05-17T04:34:41.541 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2024-05-17T04:34:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:42 smithi105 ceph-mon[34795]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:42.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:42 smithi110 ceph-mon[35497]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:42.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:42 smithi050 ceph-mon[31739]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:42.493 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-05-17T04:34:40.756027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657597,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":763,"num_read_kb":478,"num_write":303,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377764,"kb_used_data":11972,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564060,"statfs":{"total":1151940427776,"available":1151553597440,"internally_reserved":0,"allocated":12259328,"data_stored":8128793,"data_compressed":496092,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710533+0000","last_change":"2024-05-17T04:31:42.946840+0000","last_active":"2024-05-17T04:34:18.710533+0000","last_peered":"2024-05-17T04:34:18.710533+0000","last_clean":"2024-05-17T04:34:18.710533+0000","last_became_active":"2024-05-17T04:31:42.946391+0000","last_became_peered":"2024-05-17T04:31:42.946391+0000","last_unstale":"2024-05-17T04:34:18.710533+0000","last_undegraded":"2024-05-17T04:34:18.710533+0000","last_fullsized":"2024-05-17T04:34:18.710533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:06:29.922596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727177+0000","last_change":"2024-05-17T04:31:42.934805+0000","last_active":"2024-05-17T04:34:18.727177+0000","last_peered":"2024-05-17T04:34:18.727177+0000","last_clean":"2024-05-17T04:34:18.727177+0000","last_became_active":"2024-05-17T04:31:42.933527+0000","last_became_peered":"2024-05-17T04:31:42.933527+0000","last_unstale":"2024-05-17T04:34:18.727177+0000","last_undegraded":"2024-05-17T04:34:18.727177+0000","last_fullsized":"2024-05-17T04:34:18.727177+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:23:31.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728787+0000","last_change":"2024-05-17T04:31:42.935619+0000","last_active":"2024-05-17T04:34:18.728787+0000","last_peered":"2024-05-17T04:34:18.728787+0000","last_clean":"2024-05-17T04:34:18.728787+0000","last_became_active":"2024-05-17T04:31:42.935270+0000","last_became_peered":"2024-05-17T04:31:42.935270+0000","last_unstale":"2024-05-17T04:34:18.728787+0000","last_undegraded":"2024-05-17T04:34:18.728787+0000","last_fullsized":"2024-05-17T04:34:18.728787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:17.059859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710815+0000","last_change":"2024-05-17T04:31:42.928754+0000","last_active":"2024-05-17T04:34:18.710815+0000","last_peered":"2024-05-17T04:34:18.710815+0000","last_clean":"2024-05-17T04:34:18.710815+0000","last_became_active":"2024-05-17T04:31:42.927407+0000","last_became_peered":"2024-05-17T04:31:42.927407+0000","last_unstale":"2024-05-17T04:34:18.710815+0000","last_undegraded":"2024-05-17T04:34:18.710815+0000","last_fullsized":"2024-05-17T04:34:18.710815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:38:33.270224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.695374+0000","last_change":"2024-05-17T04:31:42.929405+0000","last_active":"2024-05-17T04:34:18.695374+0000","last_peered":"2024-05-17T04:34:18.695374+0000","last_clean":"2024-05-17T04:34:18.695374+0000","last_became_active":"2024-05-17T04:31:42.928970+0000","last_became_peered":"2024-05-17T04:31:42.928970+0000","last_unstale":"2024-05-17T04:34:18.695374+0000","last_undegraded":"2024-05-17T04:34:18.695374+0000","last_fullsized":"2024-05-17T04:34:18.695374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:22:39.866681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727284+0000","last_change":"2024-05-17T04:31:42.929773+0000","last_active":"2024-05-17T04:34:18.727284+0000","last_peered":"2024-05-17T04:34:18.727284+0000","last_clean":"2024-05-17T04:34:18.727284+0000","last_became_active":"2024-05-17T04:31:42.929449+0000","last_became_peered":"2024-05-17T04:31:42.929449+0000","last_unstale":"2024-05-17T04:34:18.727284+0000","last_undegraded":"2024-05-17T04:34:18.727284+0000","last_fullsized":"2024-05-17T04:34:18.727284+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:05:54.544961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722730+0000","last_change":"2024-05-17T04:31:42.930674+0000","last_active":"2024-05-17T04:34:18.722730+0000","last_peered":"2024-05-17T04:34:18.722730+0000","last_clean":"2024-05-17T04:34:18.722730+0000","last_became_active":"2024-05-17T04:31:42.929752+0000","last_became_peered":"2024-05-17T04:31:42.929752+0000","last_unstale":"2024-05-17T04:34:18.722730+0000","last_undegraded":"2024-05-17T04:34:18.722730+0000","last_fullsized":"2024-05-17T04:34:18.722730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:54:56.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719974+0000","last_change":"2024-05-17T04:31:42.941443+0000","last_active":"2024-05-17T04:34:18.719974+0000","last_peered":"2024-05-17T04:34:18.719974+0000","last_clean":"2024-05-17T04:34:18.719974+0000","last_became_active":"2024-05-17T04:31:42.941052+0000","last_became_peered":"2024-05-17T04:31:42.941052+0000","last_unstale":"2024-05-17T04:34:18.719974+0000","last_undegraded":"2024-05-17T04:34:18.719974+0000","last_fullsized":"2024-05-17T04:34:18.719974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:07.151040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723304+0000","last_change":"2024-05-17T04:31:42.938708+0000","last_active":"2024-05-17T04:34:18.723304+0000","last_peered":"2024-05-17T04:34:18.723304+0000","last_clean":"2024-05-17T04:34:18.723304+0000","last_became_active":"2024-05-17T04:31:42.938330+0000","last_became_peered":"2024-05-17T04:31:42.938330+0000","last_unstale":"2024-05-17T04:34:18.723304+0000","last_undegraded":"2024-05-17T04:34:18.723304+0000","last_fullsized":"2024-05-17T04:34:18.723304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:44:23.882565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728325+0000","last_change":"2024-05-17T04:31:42.932779+0000","last_active":"2024-05-17T04:34:18.728325+0000","last_peered":"2024-05-17T04:34:18.728325+0000","last_clean":"2024-05-17T04:34:18.728325+0000","last_became_active":"2024-05-17T04:31:42.932429+0000","last_became_peered":"2024-05-17T04:31:42.932429+0000","last_unstale":"2024-05-17T04:34:18.728325+0000","last_undegraded":"2024-05-17T04:34:18.728325+0000","last_fullsized":"2024-05-17T04:34:18.728325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:22:17.139775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724668+0000","last_change":"2024-05-17T04:31:42.932713+0000","last_active":"2024-05-17T04:34:18.724668+0000","last_peered":"2024-05-17T04:34:18.724668+0000","last_clean":"2024-05-17T04:34:18.724668+0000","last_became_active":"2024-05-17T04:31:42.931451+0000","last_became_peered":"2024-05-17T04:31:42.931451+0000","last_unstale":"2024-05-17T04:34:18.724668+0000","last_undegraded":"2024-05-17T04:34:18.724668+0000","last_fullsized":"2024-05-17T04:34:18.724668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:49:03.624279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720941+0000","last_change":"2024-05-17T04:32:20.057506+0000","last_active":"2024-05-17T04:34:18.720941+0000","last_peered":"2024-05-17T04:34:18.720941+0000","last_clean":"2024-05-17T04:34:18.720941+0000","last_became_active":"2024-05-17T04:32:20.057245+0000","last_became_peered":"2024-05-17T04:32:20.057245+0000","last_unstale":"2024-05-17T04:34:18.720941+0000","last_undegraded":"2024-05-17T04:34:18.720941+0000","last_fullsized":"2024-05-17T04:34:18.720941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:31:14.714699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727891+0000","last_change":"2024-05-17T04:31:42.930412+0000","last_active":"2024-05-17T04:34:18.727891+0000","last_peered":"2024-05-17T04:34:18.727891+0000","last_clean":"2024-05-17T04:34:18.727891+0000","last_became_active":"2024-05-17T04:31:42.929930+0000","last_became_peered":"2024-05-17T04:31:42.929930+0000","last_unstale":"2024-05-17T04:34:18.727891+0000","last_undegraded":"2024-05-17T04:34:18.727891+0000","last_fullsized":"2024-05-17T04:34:18.727891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:19:12.942064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727343+0000","last_change":"2024-05-17T04:31:42.932696+0000","last_active":"2024-05-17T04:34:18.727343+0000","last_peered":"2024-05-17T04:34:18.727343+0000","last_clean":"2024-05-17T04:34:18.727343+0000","last_became_active":"2024-05-17T04:31:42.932391+0000","last_became_peered":"2024-05-17T04:31:42.932391+0000","last_unstale":"2024-05-17T04:34:18.727343+0000","last_undegraded":"2024-05-17T04:34:18.727343+0000","last_fullsized":"2024-05-17T04:34:18.727343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:10:13.071352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.712971+0000","last_change":"2024-05-17T04:31:42.943331+0000","last_active":"2024-05-17T04:34:18.712971+0000","last_peered":"2024-05-17T04:34:18.712971+0000","last_clean":"2024-05-17T04:34:18.712971+0000","last_became_active":"2024-05-17T04:31:42.942963+0000","last_became_peered":"2024-05-17T04:31:42.942963+0000","last_unstale":"2024-05-17T04:34:18.712971+0000","last_undegraded":"2024-05-17T04:34:18.712971+0000","last_fullsized":"2024-05-17T04:34:18.712971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:27:33.614843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.726214+0000","last_change":"2024-05-17T04:31:42.935951+0000","last_active":"2024-05-17T04:34:18.726214+0000","last_peered":"2024-05-17T04:34:18.726214+0000","last_clean":"2024-05-17T04:34:18.726214+0000","last_became_active":"2024-05-17T04:31:42.935564+0000","last_became_peered":"2024-05-17T04:31:42.935564+0000","last_unstale":"2024-05-17T04:34:18.726214+0000","last_undegraded":"2024-05-17T04:34:18.726214+0000","last_fullsized":"2024-05-17T04:34:18.726214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:15:28.282780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710892+0000","last_change":"2024-05-17T04:31:42.946858+0000","last_active":"2024-05-17T04:34:18.710892+0000","last_peered":"2024-05-17T04:34:18.710892+0000","last_clean":"2024-05-17T04:34:18.710892+0000","last_became_active":"2024-05-17T04:31:42.946398+0000","last_became_peered":"2024-05-17T04:31:42.946398+0000","last_unstale":"2024-05-17T04:34:18.710892+0000","last_undegraded":"2024-05-17T04:34:18.710892+0000","last_fullsized":"2024-05-17T04:34:18.710892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:51:09.584967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714902+0000","last_change":"2024-05-17T04:31:42.934716+0000","last_active":"2024-05-17T04:34:18.714902+0000","last_peered":"2024-05-17T04:34:18.714902+0000","last_clean":"2024-05-17T04:34:18.714902+0000","last_became_active":"2024-05-17T04:31:42.934379+0000","last_became_peered":"2024-05-17T04:31:42.934379+0000","last_unstale":"2024-05-17T04:34:18.714902+0000","last_undegraded":"2024-05-17T04:34:18.714902+0000","last_fullsized":"2024-05-17T04:34:18.714902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:54:01.154473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713381+0000","last_change":"2024-05-17T04:31:42.926424+0000","last_active":"2024-05-17T04:34:18.713381+0000","last_peered":"2024-05-17T04:34:18.713381+0000","last_clean":"2024-05-17T04:34:18.713381+0000","last_became_active":"2024-05-17T04:31:42.925948+0000","last_became_peered":"2024-05-17T04:31:42.925948+0000","last_unstale":"2024-05-17T04:34:18.713381+0000","last_undegraded":"2024-05-17T04:34:18.713381+0000","last_fullsized":"2024-05-17T04:34:18.713381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:27:48.495395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933287+0000","last_change":"2024-05-17T04:31:55.992551+0000","last_active":"2024-05-17T04:33:20.933287+0000","last_peered":"2024-05-17T04:33:20.933287+0000","last_clean":"2024-05-17T04:33:20.933287+0000","last_became_active":"2024-05-17T04:31:55.992138+0000","last_became_peered":"2024-05-17T04:31:55.992138+0000","last_unstale":"2024-05-17T04:33:20.933287+0000","last_undegraded":"2024-05-17T04:33:20.933287+0000","last_fullsized":"2024-05-17T04:33:20.933287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:43:40.296492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933079+0000","last_change":"2024-05-17T04:31:39.890303+0000","last_active":"2024-05-17T04:33:20.933079+0000","last_peered":"2024-05-17T04:33:20.933079+0000","last_clean":"2024-05-17T04:33:20.933079+0000","last_became_active":"2024-05-17T04:31:39.889970+0000","last_became_peered":"2024-05-17T04:31:39.889970+0000","last_unstale":"2024-05-17T04:33:20.933079+0000","last_undegraded":"2024-05-17T04:33:20.933079+0000","last_fullsized":"2024-05-17T04:33:20.933079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:29:33.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.710993+0000","last_change":"2024-05-17T04:31:42.942203+0000","last_active":"2024-05-17T04:34:18.710993+0000","last_peered":"2024-05-17T04:34:18.710993+0000","last_clean":"2024-05-17T04:34:18.710993+0000","last_became_active":"2024-05-17T04:31:42.940563+0000","last_became_peered":"2024-05-17T04:31:42.940563+0000","last_unstale":"2024-05-17T04:34:18.710993+0000","last_undegraded":"2024-05-17T04:34:18.710993+0000","last_fullsized":"2024-05-17T04:34:18.710993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:19:10.813809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934888+0000","last_change":"2024-05-17T04:31:55.988355+0000","last_active":"2024-05-17T04:33:20.934888+0000","last_peered":"2024-05-17T04:33:20.934888+0000","last_clean":"2024-05-17T04:33:20.934888+0000","last_became_active":"2024-05-17T04:31:55.987874+0000","last_became_peered":"2024-05-17T04:31:55.987874+0000","last_unstale":"2024-05-17T04:33:20.934888+0000","last_undegraded":"2024-05-17T04:33:20.934888+0000","last_fullsized":"2024-05-17T04:33:20.934888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:13:30.650959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.101378+0000","last_change":"2024-05-17T04:31:39.896754+0000","last_active":"2024-05-17T04:33:21.101378+0000","last_peered":"2024-05-17T04:33:21.101378+0000","last_clean":"2024-05-17T04:33:21.101378+0000","last_became_active":"2024-05-17T04:31:39.896369+0000","last_became_peered":"2024-05-17T04:31:39.896369+0000","last_unstale":"2024-05-17T04:33:21.101378+0000","last_undegraded":"2024-05-17T04:33:21.101378+0000","last_fullsized":"2024-05-17T04:33:21.101378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:33:03.086963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728716+0000","last_change":"2024-05-17T04:31:42.938556+0000","last_active":"2024-05-17T04:34:18.728716+0000","last_peered":"2024-05-17T04:34:18.728716+0000","last_clean":"2024-05-17T04:34:18.728716+0000","last_became_active":"2024-05-17T04:31:42.938228+0000","last_became_peered":"2024-05-17T04:31:42.938228+0000","last_unstale":"2024-05-17T04:34:18.728716+0000","last_undegraded":"2024-05-17T04:34:18.728716+0000","last_fullsized":"2024-05-17T04:34:18.728716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:18:32.502369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935539+0000","last_change":"2024-05-17T04:31:55.991620+0000","last_active":"2024-05-17T04:33:20.935539+0000","last_peered":"2024-05-17T04:33:20.935539+0000","last_clean":"2024-05-17T04:33:20.935539+0000","last_became_active":"2024-05-17T04:31:55.990762+0000","last_became_peered":"2024-05-17T04:31:55.990762+0000","last_unstale":"2024-05-17T04:33:20.935539+0000","last_undegraded":"2024-05-17T04:33:20.935539+0000","last_fullsized":"2024-05-17T04:33:20.935539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:22:49.033835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935911+0000","last_change":"2024-05-17T04:32:20.067654+0000","last_active":"2024-05-17T04:33:20.935911+0000","last_peered":"2024-05-17T04:33:20.935911+0000","last_clean":"2024-05-17T04:33:20.935911+0000","last_became_active":"2024-05-17T04:32:20.062315+0000","last_became_peered":"2024-05-17T04:32:20.062315+0000","last_unstale":"2024-05-17T04:33:20.935911+0000","last_undegraded":"2024-05-17T04:33:20.935911+0000","last_fullsized":"2024-05-17T04:33:20.935911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:32:52.479838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.729123+0000","last_change":"2024-05-17T04:31:42.938076+0000","last_active":"2024-05-17T04:34:18.729123+0000","last_peered":"2024-05-17T04:34:18.729123+0000","last_clean":"2024-05-17T04:34:18.729123+0000","last_became_active":"2024-05-17T04:31:42.937177+0000","last_became_peered":"2024-05-17T04:31:42.937177+0000","last_unstale":"2024-05-17T04:34:18.729123+0000","last_undegraded":"2024-05-17T04:34:18.729123+0000","last_fullsized":"2024-05-17T04:34:18.729123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:08:29.759076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935296+0000","last_change":"2024-05-17T04:31:55.988774+0000","last_active":"2024-05-17T04:33:20.935296+0000","last_peered":"2024-05-17T04:33:20.935296+0000","last_clean":"2024-05-17T04:33:20.935296+0000","last_became_active":"2024-05-17T04:31:55.987934+0000","last_became_peered":"2024-05-17T04:31:55.987934+0000","last_unstale":"2024-05-17T04:33:20.935296+0000","last_undegraded":"2024-05-17T04:33:20.935296+0000","last_fullsized":"2024-05-17T04:33:20.935296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:24:53.408890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935390+0000","last_change":"2024-05-17T04:31:39.896382+0000","last_active":"2024-05-17T04:33:20.935390+0000","last_peered":"2024-05-17T04:33:20.935390+0000","last_clean":"2024-05-17T04:33:20.935390+0000","last_became_active":"2024-05-17T04:31:39.895863+0000","last_became_peered":"2024-05-17T04:31:39.895863+0000","last_unstale":"2024-05-17T04:33:20.935390+0000","last_undegraded":"2024-05-17T04:33:20.935390+0000","last_fullsized":"2024-05-17T04:33:20.935390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:08:37.587385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725449+0000","last_change":"2024-05-17T04:31:42.938624+0000","last_active":"2024-05-17T04:34:18.725449+0000","last_peered":"2024-05-17T04:34:18.725449+0000","last_clean":"2024-05-17T04:34:18.725449+0000","last_became_active":"2024-05-17T04:31:42.938197+0000","last_became_peered":"2024-05-17T04:31:42.938197+0000","last_unstale":"2024-05-17T04:34:18.725449+0000","last_undegraded":"2024-05-17T04:34:18.725449+0000","last_fullsized":"2024-05-17T04:34:18.725449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:47:14.474259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933704+0000","last_change":"2024-05-17T04:31:55.988254+0000","last_active":"2024-05-17T04:33:20.933704+0000","last_peered":"2024-05-17T04:33:20.933704+0000","last_clean":"2024-05-17T04:33:20.933704+0000","last_became_active":"2024-05-17T04:31:55.987790+0000","last_became_peered":"2024-05-17T04:31:55.987790+0000","last_unstale":"2024-05-17T04:33:20.933704+0000","last_undegraded":"2024-05-17T04:33:20.933704+0000","last_fullsized":"2024-05-17T04:33:20.933704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:07:03.189403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:37.216180+0000","last_change":"2024-05-17T04:31:39.894789+0000","last_active":"2024-05-17T04:34:37.216180+0000","last_peered":"2024-05-17T04:34:37.216180+0000","last_clean":"2024-05-17T04:34:37.216180+0000","last_became_active":"2024-05-17T04:31:39.894404+0000","last_became_peered":"2024-05-17T04:31:39.894404+0000","last_unstale":"2024-05-17T04:34:37.216180+0000","last_undegraded":"2024-05-17T04:34:37.216180+0000","last_fullsized":"2024-05-17T04:34:37.216180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:00:25.806236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.707239+0000","last_change":"2024-05-17T04:31:42.932430+0000","last_active":"2024-05-17T04:34:18.707239+0000","last_peered":"2024-05-17T04:34:18.707239+0000","last_clean":"2024-05-17T04:34:18.707239+0000","last_became_active":"2024-05-17T04:31:42.932039+0000","last_became_peered":"2024-05-17T04:31:42.932039+0000","last_unstale":"2024-05-17T04:34:18.707239+0000","last_undegraded":"2024-05-17T04:34:18.707239+0000","last_fullsized":"2024-05-17T04:34:18.707239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:46:53.928092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933905+0000","last_change":"2024-05-17T04:31:55.992898+0000","last_active":"2024-05-17T04:33:20.933905+0000","last_peered":"2024-05-17T04:33:20.933905+0000","last_clean":"2024-05-17T04:33:20.933905+0000","last_became_active":"2024-05-17T04:31:55.992459+0000","last_became_peered":"2024-05-17T04:31:55.992459+0000","last_unstale":"2024-05-17T04:33:20.933905+0000","last_undegraded":"2024-05-17T04:33:20.933905+0000","last_fullsized":"2024-05-17T04:33:20.933905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:45:24.888533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.104751+0000","last_change":"2024-05-17T04:31:39.893014+0000","last_active":"2024-05-17T04:33:21.104751+0000","last_peered":"2024-05-17T04:33:21.104751+0000","last_clean":"2024-05-17T04:33:21.104751+0000","last_became_active":"2024-05-17T04:31:39.892602+0000","last_became_peered":"2024-05-17T04:31:39.892602+0000","last_unstale":"2024-05-17T04:33:21.104751+0000","last_undegraded":"2024-05-17T04:33:21.104751+0000","last_fullsized":"2024-05-17T04:33:21.104751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:45:12.742516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722193+0000","last_change":"2024-05-17T04:31:42.945750+0000","last_active":"2024-05-17T04:34:18.722193+0000","last_peered":"2024-05-17T04:34:18.722193+0000","last_clean":"2024-05-17T04:34:18.722193+0000","last_became_active":"2024-05-17T04:31:42.945332+0000","last_became_peered":"2024-05-17T04:31:42.945332+0000","last_unstale":"2024-05-17T04:34:18.722193+0000","last_undegraded":"2024-05-17T04:34:18.722193+0000","last_fullsized":"2024-05-17T04:34:18.722193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:36:00.521489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593707+0000","last_change":"2024-05-17T04:31:55.984513+0000","last_active":"2024-05-17T04:33:21.593707+0000","last_peered":"2024-05-17T04:33:21.593707+0000","last_clean":"2024-05-17T04:33:21.593707+0000","last_became_active":"2024-05-17T04:31:55.984065+0000","last_became_peered":"2024-05-17T04:31:55.984065+0000","last_unstale":"2024-05-17T04:33:21.593707+0000","last_undegraded":"2024-05-17T04:33:21.593707+0000","last_fullsized":"2024-05-17T04:33:21.593707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:07:06.370969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.105961+0000","last_change":"2024-05-17T04:31:39.889551+0000","last_active":"2024-05-17T04:33:21.105961+0000","last_peered":"2024-05-17T04:33:21.105961+0000","last_clean":"2024-05-17T04:33:21.105961+0000","last_became_active":"2024-05-17T04:31:39.889150+0000","last_became_peered":"2024-05-17T04:31:39.889150+0000","last_unstale":"2024-05-17T04:33:21.105961+0000","last_undegraded":"2024-05-17T04:33:21.105961+0000","last_fullsized":"2024-05-17T04:33:21.105961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:14:06.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717119+0000","last_change":"2024-05-17T04:31:42.922236+0000","last_active":"2024-05-17T04:34:18.717119+0000","last_peered":"2024-05-17T04:34:18.717119+0000","last_clean":"2024-05-17T04:34:18.717119+0000","last_became_active":"2024-05-17T04:31:42.921907+0000","last_became_peered":"2024-05-17T04:31:42.921907+0000","last_unstale":"2024-05-17T04:34:18.717119+0000","last_undegraded":"2024-05-17T04:34:18.717119+0000","last_fullsized":"2024-05-17T04:34:18.717119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:10:09.941419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593667+0000","last_change":"2024-05-17T04:31:55.997210+0000","last_active":"2024-05-17T04:33:21.593667+0000","last_peered":"2024-05-17T04:33:21.593667+0000","last_clean":"2024-05-17T04:33:21.593667+0000","last_became_active":"2024-05-17T04:31:55.996795+0000","last_became_peered":"2024-05-17T04:31:55.996795+0000","last_unstale":"2024-05-17T04:33:21.593667+0000","last_undegraded":"2024-05-17T04:33:21.593667+0000","last_fullsized":"2024-05-17T04:33:21.593667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:51:42.271149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107885+0000","last_change":"2024-05-17T04:31:39.891980+0000","last_active":"2024-05-17T04:33:21.107885+0000","last_peered":"2024-05-17T04:33:21.107885+0000","last_clean":"2024-05-17T04:33:21.107885+0000","last_became_active":"2024-05-17T04:31:39.891636+0000","last_became_peered":"2024-05-17T04:31:39.891636+0000","last_unstale":"2024-05-17T04:33:21.107885+0000","last_undegraded":"2024-05-17T04:33:21.107885+0000","last_fullsized":"2024-05-17T04:33:21.107885+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:05:33.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720151+0000","last_change":"2024-05-17T04:31:42.929970+0000","last_active":"2024-05-17T04:34:18.720151+0000","last_peered":"2024-05-17T04:34:18.720151+0000","last_clean":"2024-05-17T04:34:18.720151+0000","last_became_active":"2024-05-17T04:31:42.929587+0000","last_became_peered":"2024-05-17T04:31:42.929587+0000","last_unstale":"2024-05-17T04:34:18.720151+0000","last_undegraded":"2024-05-17T04:34:18.720151+0000","last_fullsized":"2024-05-17T04:34:18.720151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:05:38.039359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100781+0000","last_change":"2024-05-17T04:31:55.991388+0000","last_active":"2024-05-17T04:33:21.100781+0000","last_peered":"2024-05-17T04:33:21.100781+0000","last_clean":"2024-05-17T04:33:21.100781+0000","last_became_active":"2024-05-17T04:31:55.990979+0000","last_became_peered":"2024-05-17T04:31:55.990979+0000","last_unstale":"2024-05-17T04:33:21.100781+0000","last_undegraded":"2024-05-17T04:33:21.100781+0000","last_fullsized":"2024-05-17T04:33:21.100781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:55:34.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:38.216925+0000","last_change":"2024-05-17T04:31:39.893076+0000","last_active":"2024-05-17T04:34:38.216925+0000","last_peered":"2024-05-17T04:34:38.216925+0000","last_clean":"2024-05-17T04:34:38.216925+0000","last_became_active":"2024-05-17T04:31:39.892683+0000","last_became_peered":"2024-05-17T04:31:39.892683+0000","last_unstale":"2024-05-17T04:34:38.216925+0000","last_undegraded":"2024-05-17T04:34:38.216925+0000","last_fullsized":"2024-05-17T04:34:38.216925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:32:19.583662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.730271+0000","last_change":"2024-05-17T04:31:42.938431+0000","last_active":"2024-05-17T04:34:18.730271+0000","last_peered":"2024-05-17T04:34:18.730271+0000","last_clean":"2024-05-17T04:34:18.730271+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:34:18.730271+0000","last_undegraded":"2024-05-17T04:34:18.730271+0000","last_fullsized":"2024-05-17T04:34:18.730271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:56:34.505923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934775+0000","last_change":"2024-05-17T04:32:20.065723+0000","last_active":"2024-05-17T04:33:20.934775+0000","last_peered":"2024-05-17T04:33:20.934775+0000","last_clean":"2024-05-17T04:33:20.934775+0000","last_became_active":"2024-05-17T04:32:20.065266+0000","last_became_peered":"2024-05-17T04:32:20.065266+0000","last_unstale":"2024-05-17T04:33:20.934775+0000","last_undegraded":"2024-05-17T04:33:20.934775+0000","last_fullsized":"2024-05-17T04:33:20.934775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:16:54.256206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935134+0000","last_change":"2024-05-17T04:31:39.892324+0000","last_active":"2024-05-17T04:33:20.935134+0000","last_peered":"2024-05-17T04:33:20.935134+0000","last_clean":"2024-05-17T04:33:20.935134+0000","last_became_active":"2024-05-17T04:31:39.891398+0000","last_became_peered":"2024-05-17T04:31:39.891398+0000","last_unstale":"2024-05-17T04:33:20.935134+0000","last_undegraded":"2024-05-17T04:33:20.935134+0000","last_fullsized":"2024-05-17T04:33:20.935134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:00:06.423894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.721820+0000","last_change":"2024-05-17T04:31:42.948320+0000","last_active":"2024-05-17T04:34:18.721820+0000","last_peered":"2024-05-17T04:34:18.721820+0000","last_clean":"2024-05-17T04:34:18.721820+0000","last_became_active":"2024-05-17T04:31:42.948009+0000","last_became_peered":"2024-05-17T04:31:42.948009+0000","last_unstale":"2024-05-17T04:34:18.721820+0000","last_undegraded":"2024-05-17T04:34:18.721820+0000","last_fullsized":"2024-05-17T04:34:18.721820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:51:28.464925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935832+0000","last_change":"2024-05-17T04:31:55.984602+0000","last_active":"2024-05-17T04:33:20.935832+0000","last_peered":"2024-05-17T04:33:20.935832+0000","last_clean":"2024-05-17T04:33:20.935832+0000","last_became_active":"2024-05-17T04:31:55.984152+0000","last_became_peered":"2024-05-17T04:31:55.984152+0000","last_unstale":"2024-05-17T04:33:20.935832+0000","last_undegraded":"2024-05-17T04:33:20.935832+0000","last_fullsized":"2024-05-17T04:33:20.935832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:31:23.456878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108324+0000","last_change":"2024-05-17T04:31:39.896889+0000","last_active":"2024-05-17T04:33:21.108324+0000","last_peered":"2024-05-17T04:33:21.108324+0000","last_clean":"2024-05-17T04:33:21.108324+0000","last_became_active":"2024-05-17T04:31:39.896516+0000","last_became_peered":"2024-05-17T04:31:39.896516+0000","last_unstale":"2024-05-17T04:33:21.108324+0000","last_undegraded":"2024-05-17T04:33:21.108324+0000","last_fullsized":"2024-05-17T04:33:21.108324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:45:46.214589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724108+0000","last_change":"2024-05-17T04:31:42.928588+0000","last_active":"2024-05-17T04:34:18.724108+0000","last_peered":"2024-05-17T04:34:18.724108+0000","last_clean":"2024-05-17T04:34:18.724108+0000","last_became_active":"2024-05-17T04:31:42.928215+0000","last_became_peered":"2024-05-17T04:31:42.928215+0000","last_unstale":"2024-05-17T04:34:18.724108+0000","last_undegraded":"2024-05-17T04:34:18.724108+0000","last_fullsized":"2024-05-17T04:34:18.724108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:49:31.894516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.104857+0000","last_change":"2024-05-17T04:32:20.063156+0000","last_active":"2024-05-17T04:33:21.104857+0000","last_peered":"2024-05-17T04:33:21.104857+0000","last_clean":"2024-05-17T04:33:21.104857+0000","last_became_active":"2024-05-17T04:32:20.062600+0000","last_became_peered":"2024-05-17T04:32:20.062600+0000","last_unstale":"2024-05-17T04:33:21.104857+0000","last_undegraded":"2024-05-17T04:33:21.104857+0000","last_fullsized":"2024-05-17T04:33:21.104857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:58:04.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:39.217946+0000","last_change":"2024-05-17T04:31:39.896849+0000","last_active":"2024-05-17T04:34:39.217946+0000","last_peered":"2024-05-17T04:34:39.217946+0000","last_clean":"2024-05-17T04:34:39.217946+0000","last_became_active":"2024-05-17T04:31:39.896134+0000","last_became_peered":"2024-05-17T04:31:39.896134+0000","last_unstale":"2024-05-17T04:34:39.217946+0000","last_undegraded":"2024-05-17T04:34:39.217946+0000","last_fullsized":"2024-05-17T04:34:39.217946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:00:27.150148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":230,"num_read_kb":176,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.711508+0000","last_change":"2024-05-17T04:31:42.932473+0000","last_active":"2024-05-17T04:34:18.711508+0000","last_peered":"2024-05-17T04:34:18.711508+0000","last_clean":"2024-05-17T04:34:18.711508+0000","last_became_active":"2024-05-17T04:31:42.932120+0000","last_became_peered":"2024-05-17T04:31:42.932120+0000","last_unstale":"2024-05-17T04:34:18.711508+0000","last_undegraded":"2024-05-17T04:34:18.711508+0000","last_fullsized":"2024-05-17T04:34:18.711508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:57:35.056071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108802+0000","last_change":"2024-05-17T04:31:55.999678+0000","last_active":"2024-05-17T04:33:21.108802+0000","last_peered":"2024-05-17T04:33:21.108802+0000","last_clean":"2024-05-17T04:33:21.108802+0000","last_became_active":"2024-05-17T04:31:55.999236+0000","last_became_peered":"2024-05-17T04:31:55.999236+0000","last_unstale":"2024-05-17T04:33:21.108802+0000","last_undegraded":"2024-05-17T04:33:21.108802+0000","last_fullsized":"2024-05-17T04:33:21.108802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:16:27.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933560+0000","last_change":"2024-05-17T04:31:39.894334+0000","last_active":"2024-05-17T04:33:20.933560+0000","last_peered":"2024-05-17T04:33:20.933560+0000","last_clean":"2024-05-17T04:33:20.933560+0000","last_became_active":"2024-05-17T04:31:39.893801+0000","last_became_peered":"2024-05-17T04:31:39.893801+0000","last_unstale":"2024-05-17T04:33:20.933560+0000","last_undegraded":"2024-05-17T04:33:20.933560+0000","last_fullsized":"2024-05-17T04:33:20.933560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:26:07.143110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.712085+0000","last_change":"2024-05-17T04:31:42.934598+0000","last_active":"2024-05-17T04:34:18.712085+0000","last_peered":"2024-05-17T04:34:18.712085+0000","last_clean":"2024-05-17T04:34:18.712085+0000","last_became_active":"2024-05-17T04:31:42.934268+0000","last_became_peered":"2024-05-17T04:31:42.934268+0000","last_unstale":"2024-05-17T04:34:18.712085+0000","last_undegraded":"2024-05-17T04:34:18.712085+0000","last_fullsized":"2024-05-17T04:34:18.712085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T09:50:13.868834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593396+0000","last_change":"2024-05-17T04:31:55.986888+0000","last_active":"2024-05-17T04:33:21.593396+0000","last_peered":"2024-05-17T04:33:21.593396+0000","last_clean":"2024-05-17T04:33:21.593396+0000","last_became_active":"2024-05-17T04:31:55.986369+0000","last_became_peered":"2024-05-17T04:31:55.986369+0000","last_unstale":"2024-05-17T04:33:21.593396+0000","last_undegraded":"2024-05-17T04:33:21.593396+0000","last_fullsized":"2024-05-17T04:33:21.593396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:42:40.672724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108168+0000","last_change":"2024-05-17T04:31:39.891226+0000","last_active":"2024-05-17T04:33:21.108168+0000","last_peered":"2024-05-17T04:33:21.108168+0000","last_clean":"2024-05-17T04:33:21.108168+0000","last_became_active":"2024-05-17T04:31:39.890376+0000","last_became_peered":"2024-05-17T04:31:39.890376+0000","last_unstale":"2024-05-17T04:33:21.108168+0000","last_undegraded":"2024-05-17T04:33:21.108168+0000","last_fullsized":"2024-05-17T04:33:21.108168+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:13:14.749546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.724900+0000","last_change":"2024-05-17T04:32:20.062998+0000","last_active":"2024-05-17T04:34:18.724900+0000","last_peered":"2024-05-17T04:34:18.724900+0000","last_clean":"2024-05-17T04:34:18.724900+0000","last_became_active":"2024-05-17T04:32:20.062485+0000","last_became_peered":"2024-05-17T04:32:20.062485+0000","last_unstale":"2024-05-17T04:34:18.724900+0000","last_undegraded":"2024-05-17T04:34:18.724900+0000","last_fullsized":"2024-05-17T04:34:18.724900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:20:23.822735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.932923+0000","last_change":"2024-05-17T04:31:55.994500+0000","last_active":"2024-05-17T04:33:20.932923+0000","last_peered":"2024-05-17T04:33:20.932923+0000","last_clean":"2024-05-17T04:33:20.932923+0000","last_became_active":"2024-05-17T04:31:55.994130+0000","last_became_peered":"2024-05-17T04:31:55.994130+0000","last_unstale":"2024-05-17T04:33:20.932923+0000","last_undegraded":"2024-05-17T04:33:20.932923+0000","last_fullsized":"2024-05-17T04:33:20.932923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:15:33.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:38.217166+0000","last_change":"2024-05-17T04:31:39.894453+0000","last_active":"2024-05-17T04:34:38.217166+0000","last_peered":"2024-05-17T04:34:38.217166+0000","last_clean":"2024-05-17T04:34:38.217166+0000","last_became_active":"2024-05-17T04:31:39.893628+0000","last_became_peered":"2024-05-17T04:31:39.893628+0000","last_unstale":"2024-05-17T04:34:38.217166+0000","last_undegraded":"2024-05-17T04:34:38.217166+0000","last_fullsized":"2024-05-17T04:34:38.217166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:55:48.098182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722485+0000","last_change":"2024-05-17T04:31:42.939487+0000","last_active":"2024-05-17T04:34:18.722485+0000","last_peered":"2024-05-17T04:34:18.722485+0000","last_clean":"2024-05-17T04:34:18.722485+0000","last_became_active":"2024-05-17T04:31:42.938719+0000","last_became_peered":"2024-05-17T04:31:42.938719+0000","last_unstale":"2024-05-17T04:34:18.722485+0000","last_undegraded":"2024-05-17T04:34:18.722485+0000","last_fullsized":"2024-05-17T04:34:18.722485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:22:42.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106435+0000","last_change":"2024-05-17T04:31:55.998287+0000","last_active":"2024-05-17T04:33:21.106435+0000","last_peered":"2024-05-17T04:33:21.106435+0000","last_clean":"2024-05-17T04:33:21.106435+0000","last_became_active":"2024-05-17T04:31:55.997876+0000","last_became_peered":"2024-05-17T04:31:55.997876+0000","last_unstale":"2024-05-17T04:33:21.106435+0000","last_undegraded":"2024-05-17T04:33:21.106435+0000","last_fullsized":"2024-05-17T04:33:21.106435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:20.523375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934869+0000","last_change":"2024-05-17T04:31:39.896995+0000","last_active":"2024-05-17T04:33:20.934869+0000","last_peered":"2024-05-17T04:33:20.934869+0000","last_clean":"2024-05-17T04:33:20.934869+0000","last_became_active":"2024-05-17T04:31:39.896607+0000","last_became_peered":"2024-05-17T04:31:39.896607+0000","last_unstale":"2024-05-17T04:33:20.934869+0000","last_undegraded":"2024-05-17T04:33:20.934869+0000","last_fullsized":"2024-05-17T04:33:20.934869+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:13:26.761427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714158+0000","last_change":"2024-05-17T04:32:20.061507+0000","last_active":"2024-05-17T04:34:18.714158+0000","last_peered":"2024-05-17T04:34:18.714158+0000","last_clean":"2024-05-17T04:34:18.714158+0000","last_became_active":"2024-05-17T04:32:20.061245+0000","last_became_peered":"2024-05-17T04:32:20.061245+0000","last_unstale":"2024-05-17T04:34:18.714158+0000","last_undegraded":"2024-05-17T04:34:18.714158+0000","last_fullsized":"2024-05-17T04:34:18.714158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:54:43.800524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107875+0000","last_change":"2024-05-17T04:31:55.991340+0000","last_active":"2024-05-17T04:33:21.107875+0000","last_peered":"2024-05-17T04:33:21.107875+0000","last_clean":"2024-05-17T04:33:21.107875+0000","last_became_active":"2024-05-17T04:31:55.990733+0000","last_became_peered":"2024-05-17T04:31:55.990733+0000","last_unstale":"2024-05-17T04:33:21.107875+0000","last_undegraded":"2024-05-17T04:33:21.107875+0000","last_fullsized":"2024-05-17T04:33:21.107875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:01:47.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593625+0000","last_change":"2024-05-17T04:32:20.070153+0000","last_active":"2024-05-17T04:33:21.593625+0000","last_peered":"2024-05-17T04:33:21.593625+0000","last_clean":"2024-05-17T04:33:21.593625+0000","last_became_active":"2024-05-17T04:32:20.062400+0000","last_became_peered":"2024-05-17T04:32:20.062400+0000","last_unstale":"2024-05-17T04:33:21.593625+0000","last_undegraded":"2024-05-17T04:33:21.593625+0000","last_fullsized":"2024-05-17T04:33:21.593625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:25:03.332079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714322+0000","last_change":"2024-05-17T04:32:20.064121+0000","last_active":"2024-05-17T04:34:18.714322+0000","last_peered":"2024-05-17T04:34:18.714322+0000","last_clean":"2024-05-17T04:34:18.714322+0000","last_became_active":"2024-05-17T04:32:20.063707+0000","last_became_peered":"2024-05-17T04:32:20.063707+0000","last_unstale":"2024-05-17T04:34:18.714322+0000","last_undegraded":"2024-05-17T04:34:18.714322+0000","last_fullsized":"2024-05-17T04:34:18.714322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:57:45.362980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934905+0000","last_change":"2024-05-17T04:31:56.178977+0000","last_active":"2024-05-17T04:33:20.934905+0000","last_peered":"2024-05-17T04:33:20.934905+0000","last_clean":"2024-05-17T04:33:20.934905+0000","last_became_active":"2024-05-17T04:31:56.178452+0000","last_became_peered":"2024-05-17T04:31:56.178452+0000","last_unstale":"2024-05-17T04:33:20.934905+0000","last_undegraded":"2024-05-17T04:33:20.934905+0000","last_fullsized":"2024-05-17T04:33:20.934905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:14:19.550645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.932762+0000","last_change":"2024-05-17T04:31:39.897118+0000","last_active":"2024-05-17T04:33:20.932762+0000","last_peered":"2024-05-17T04:33:20.932762+0000","last_clean":"2024-05-17T04:33:20.932762+0000","last_became_active":"2024-05-17T04:31:39.896761+0000","last_became_peered":"2024-05-17T04:31:39.896761+0000","last_unstale":"2024-05-17T04:33:20.932762+0000","last_undegraded":"2024-05-17T04:33:20.932762+0000","last_fullsized":"2024-05-17T04:33:20.932762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:41:50.575131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728124+0000","last_change":"2024-05-17T04:31:42.930510+0000","last_active":"2024-05-17T04:34:18.728124+0000","last_peered":"2024-05-17T04:34:18.728124+0000","last_clean":"2024-05-17T04:34:18.728124+0000","last_became_active":"2024-05-17T04:31:42.930168+0000","last_became_peered":"2024-05-17T04:31:42.930168+0000","last_unstale":"2024-05-17T04:34:18.728124+0000","last_undegraded":"2024-05-17T04:34:18.728124+0000","last_fullsized":"2024-05-17T04:34:18.728124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:18:23.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106222+0000","last_change":"2024-05-17T04:31:55.996783+0000","last_active":"2024-05-17T04:33:21.106222+0000","last_peered":"2024-05-17T04:33:21.106222+0000","last_clean":"2024-05-17T04:33:21.106222+0000","last_became_active":"2024-05-17T04:31:55.996343+0000","last_became_peered":"2024-05-17T04:31:55.996343+0000","last_unstale":"2024-05-17T04:33:21.106222+0000","last_undegraded":"2024-05-17T04:33:21.106222+0000","last_fullsized":"2024-05-17T04:33:21.106222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:41:49.074496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934474+0000","last_change":"2024-05-17T04:31:39.890854+0000","last_active":"2024-05-17T04:33:20.934474+0000","last_peered":"2024-05-17T04:33:20.934474+0000","last_clean":"2024-05-17T04:33:20.934474+0000","last_became_active":"2024-05-17T04:31:39.890379+0000","last_became_peered":"2024-05-17T04:31:39.890379+0000","last_unstale":"2024-05-17T04:33:20.934474+0000","last_undegraded":"2024-05-17T04:33:20.934474+0000","last_fullsized":"2024-05-17T04:33:20.934474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:34:06.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.727953+0000","last_change":"2024-05-17T04:31:42.936992+0000","last_active":"2024-05-17T04:34:18.727953+0000","last_peered":"2024-05-17T04:34:18.727953+0000","last_clean":"2024-05-17T04:34:18.727953+0000","last_became_active":"2024-05-17T04:31:42.936612+0000","last_became_peered":"2024-05-17T04:31:42.936612+0000","last_unstale":"2024-05-17T04:34:18.727953+0000","last_undegraded":"2024-05-17T04:34:18.727953+0000","last_fullsized":"2024-05-17T04:34:18.727953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:27:33.067173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106256+0000","last_change":"2024-05-17T04:31:56.178921+0000","last_active":"2024-05-17T04:33:21.106256+0000","last_peered":"2024-05-17T04:33:21.106256+0000","last_clean":"2024-05-17T04:33:21.106256+0000","last_became_active":"2024-05-17T04:31:56.178430+0000","last_became_peered":"2024-05-17T04:31:56.178430+0000","last_unstale":"2024-05-17T04:33:21.106256+0000","last_undegraded":"2024-05-17T04:33:21.106256+0000","last_fullsized":"2024-05-17T04:33:21.106256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:56:37.980197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722951+0000","last_change":"2024-05-17T04:31:42.938586+0000","last_active":"2024-05-17T04:34:18.722951+0000","last_peered":"2024-05-17T04:34:18.722951+0000","last_clean":"2024-05-17T04:34:18.722951+0000","last_became_active":"2024-05-17T04:31:42.937861+0000","last_became_peered":"2024-05-17T04:31:42.937861+0000","last_unstale":"2024-05-17T04:34:18.722951+0000","last_undegraded":"2024-05-17T04:34:18.722951+0000","last_fullsized":"2024-05-17T04:34:18.722951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:03:11.790930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:34.211105+0000","last_change":"2024-05-17T04:31:39.896824+0000","last_active":"2024-05-17T04:34:34.211105+0000","last_peered":"2024-05-17T04:34:34.211105+0000","last_clean":"2024-05-17T04:34:34.211105+0000","last_became_active":"2024-05-17T04:31:39.896463+0000","last_became_peered":"2024-05-17T04:31:39.896463+0000","last_unstale":"2024-05-17T04:34:34.211105+0000","last_undegraded":"2024-05-17T04:34:34.211105+0000","last_fullsized":"2024-05-17T04:34:34.211105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:36:08.351784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934787+0000","last_change":"2024-05-17T04:31:55.993336+0000","last_active":"2024-05-17T04:33:20.934787+0000","last_peered":"2024-05-17T04:33:20.934787+0000","last_clean":"2024-05-17T04:33:20.934787+0000","last_became_active":"2024-05-17T04:31:55.992851+0000","last_became_peered":"2024-05-17T04:31:55.992851+0000","last_unstale":"2024-05-17T04:33:20.934787+0000","last_undegraded":"2024-05-17T04:33:20.934787+0000","last_fullsized":"2024-05-17T04:33:20.934787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:07:47.563166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.728581+0000","last_change":"2024-05-17T04:31:42.941862+0000","last_active":"2024-05-17T04:34:18.728581+0000","last_peered":"2024-05-17T04:34:18.728581+0000","last_clean":"2024-05-17T04:34:18.728581+0000","last_became_active":"2024-05-17T04:31:42.940304+0000","last_became_peered":"2024-05-17T04:31:42.940304+0000","last_unstale":"2024-05-17T04:34:18.728581+0000","last_undegraded":"2024-05-17T04:34:18.728581+0000","last_fullsized":"2024-05-17T04:34:18.728581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:46:14.569208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935685+0000","last_change":"2024-05-17T04:31:39.894080+0000","last_active":"2024-05-17T04:33:20.935685+0000","last_peered":"2024-05-17T04:33:20.935685+0000","last_clean":"2024-05-17T04:33:20.935685+0000","last_became_active":"2024-05-17T04:31:39.893646+0000","last_became_peered":"2024-05-17T04:31:39.893646+0000","last_unstale":"2024-05-17T04:33:20.935685+0000","last_undegraded":"2024-05-17T04:33:20.935685+0000","last_fullsized":"2024-05-17T04:33:20.935685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:27:59.844497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934014+0000","last_change":"2024-05-17T04:30:11.017843+0000","last_active":"2024-05-17T04:33:20.934014+0000","last_peered":"2024-05-17T04:33:20.934014+0000","last_clean":"2024-05-17T04:33:20.934014+0000","last_became_active":"2024-05-17T04:30:11.013443+0000","last_became_peered":"2024-05-17T04:30:11.013443+0000","last_unstale":"2024-05-17T04:33:20.934014+0000","last_undegraded":"2024-05-17T04:33:20.934014+0000","last_fullsized":"2024-05-17T04:33:20.934014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:26:43.066168+0000","last_clean_scrub_stamp":"2024-05-17T04:26:43.066168+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:27:18.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108420+0000","last_change":"2024-05-17T04:31:56.179071+0000","last_active":"2024-05-17T04:33:21.108420+0000","last_peered":"2024-05-17T04:33:21.108420+0000","last_clean":"2024-05-17T04:33:21.108420+0000","last_became_active":"2024-05-17T04:31:56.178500+0000","last_became_peered":"2024-05-17T04:31:56.178500+0000","last_unstale":"2024-05-17T04:33:21.108420+0000","last_undegraded":"2024-05-17T04:33:21.108420+0000","last_fullsized":"2024-05-17T04:33:21.108420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:48:02.144119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.709620+0000","last_change":"2024-05-17T04:31:42.932232+0000","last_active":"2024-05-17T04:34:18.709620+0000","last_peered":"2024-05-17T04:34:18.709620+0000","last_clean":"2024-05-17T04:34:18.709620+0000","last_became_active":"2024-05-17T04:31:42.931915+0000","last_became_peered":"2024-05-17T04:31:42.931915+0000","last_unstale":"2024-05-17T04:34:18.709620+0000","last_undegraded":"2024-05-17T04:34:18.709620+0000","last_fullsized":"2024-05-17T04:34:18.709620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:06:57.693180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.934214+0000","last_change":"2024-05-17T04:31:39.890852+0000","last_active":"2024-05-17T04:33:20.934214+0000","last_peered":"2024-05-17T04:33:20.934214+0000","last_clean":"2024-05-17T04:33:20.934214+0000","last_became_active":"2024-05-17T04:31:39.890417+0000","last_became_peered":"2024-05-17T04:31:39.890417+0000","last_unstale":"2024-05-17T04:33:20.934214+0000","last_undegraded":"2024-05-17T04:33:20.934214+0000","last_fullsized":"2024-05-17T04:33:20.934214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:45:47.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593365+0000","last_change":"2024-05-17T04:31:55.988931+0000","last_active":"2024-05-17T04:33:21.593365+0000","last_peered":"2024-05-17T04:33:21.593365+0000","last_clean":"2024-05-17T04:33:21.593365+0000","last_became_active":"2024-05-17T04:31:55.988366+0000","last_became_peered":"2024-05-17T04:31:55.988366+0000","last_unstale":"2024-05-17T04:33:21.593365+0000","last_undegraded":"2024-05-17T04:33:21.593365+0000","last_fullsized":"2024-05-17T04:33:21.593365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:36:30.684945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593513+0000","last_change":"2024-05-17T04:32:20.059700+0000","last_active":"2024-05-17T04:33:21.593513+0000","last_peered":"2024-05-17T04:33:21.593513+0000","last_clean":"2024-05-17T04:33:21.593513+0000","last_became_active":"2024-05-17T04:32:20.059234+0000","last_became_peered":"2024-05-17T04:32:20.059234+0000","last_unstale":"2024-05-17T04:33:21.593513+0000","last_undegraded":"2024-05-17T04:33:21.593513+0000","last_fullsized":"2024-05-17T04:33:21.593513+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:25:47.214832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.706850+0000","last_change":"2024-05-17T04:31:42.926078+0000","last_active":"2024-05-17T04:34:18.706850+0000","last_peered":"2024-05-17T04:34:18.706850+0000","last_clean":"2024-05-17T04:34:18.706850+0000","last_became_active":"2024-05-17T04:31:42.925755+0000","last_became_peered":"2024-05-17T04:31:42.925755+0000","last_unstale":"2024-05-17T04:34:18.706850+0000","last_undegraded":"2024-05-17T04:34:18.706850+0000","last_fullsized":"2024-05-17T04:34:18.706850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:30:16.952810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100687+0000","last_change":"2024-05-17T04:31:56.178973+0000","last_active":"2024-05-17T04:33:21.100687+0000","last_peered":"2024-05-17T04:33:21.100687+0000","last_clean":"2024-05-17T04:33:21.100687+0000","last_became_active":"2024-05-17T04:31:56.178359+0000","last_became_peered":"2024-05-17T04:31:56.178359+0000","last_unstale":"2024-05-17T04:33:21.100687+0000","last_undegraded":"2024-05-17T04:33:21.100687+0000","last_fullsized":"2024-05-17T04:33:21.100687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:22:48.921778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:53.769588+0000","last_change":"2024-05-17T04:31:39.887364+0000","last_active":"2024-05-17T04:33:53.769588+0000","last_peered":"2024-05-17T04:33:53.769588+0000","last_clean":"2024-05-17T04:33:53.769588+0000","last_became_active":"2024-05-17T04:31:39.886961+0000","last_became_peered":"2024-05-17T04:31:39.886961+0000","last_unstale":"2024-05-17T04:33:53.769588+0000","last_undegraded":"2024-05-17T04:33:53.769588+0000","last_fullsized":"2024-05-17T04:33:53.769588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:48:20.417903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.722555+0000","last_change":"2024-05-17T04:31:42.933338+0000","last_active":"2024-05-17T04:34:18.722555+0000","last_peered":"2024-05-17T04:34:18.722555+0000","last_clean":"2024-05-17T04:34:18.722555+0000","last_became_active":"2024-05-17T04:31:42.931619+0000","last_became_peered":"2024-05-17T04:31:42.931619+0000","last_unstale":"2024-05-17T04:34:18.722555+0000","last_undegraded":"2024-05-17T04:34:18.722555+0000","last_fullsized":"2024-05-17T04:34:18.722555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:24:52.565132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933050+0000","last_change":"2024-05-17T04:31:55.990398+0000","last_active":"2024-05-17T04:33:20.933050+0000","last_peered":"2024-05-17T04:33:20.933050+0000","last_clean":"2024-05-17T04:33:20.933050+0000","last_became_active":"2024-05-17T04:31:55.989595+0000","last_became_peered":"2024-05-17T04:31:55.989595+0000","last_unstale":"2024-05-17T04:33:20.933050+0000","last_undegraded":"2024-05-17T04:33:20.933050+0000","last_fullsized":"2024-05-17T04:33:20.933050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:41:37.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.936077+0000","last_change":"2024-05-17T04:31:40.443503+0000","last_active":"2024-05-17T04:33:20.936077+0000","last_peered":"2024-05-17T04:33:20.936077+0000","last_clean":"2024-05-17T04:33:20.936077+0000","last_became_active":"2024-05-17T04:31:40.443094+0000","last_became_peered":"2024-05-17T04:31:40.443094+0000","last_unstale":"2024-05-17T04:33:20.936077+0000","last_undegraded":"2024-05-17T04:33:20.936077+0000","last_fullsized":"2024-05-17T04:33:20.936077+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:03:26.837211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723515+0000","last_change":"2024-05-17T04:31:42.933715+0000","last_active":"2024-05-17T04:34:18.723515+0000","last_peered":"2024-05-17T04:34:18.723515+0000","last_clean":"2024-05-17T04:34:18.723515+0000","last_became_active":"2024-05-17T04:31:42.933360+0000","last_became_peered":"2024-05-17T04:31:42.933360+0000","last_unstale":"2024-05-17T04:34:18.723515+0000","last_undegraded":"2024-05-17T04:34:18.723515+0000","last_fullsized":"2024-05-17T04:34:18.723515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T11:02:30.568910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107267+0000","last_change":"2024-05-17T04:33:21.106627+0000","last_active":"2024-05-17T04:33:21.107267+0000","last_peered":"2024-05-17T04:33:21.107267+0000","last_clean":"2024-05-17T04:33:21.107267+0000","last_became_active":"2024-05-17T04:33:21.106124+0000","last_became_peered":"2024-05-17T04:33:21.106124+0000","last_unstale":"2024-05-17T04:33:21.107267+0000","last_undegraded":"2024-05-17T04:33:21.107267+0000","last_fullsized":"2024-05-17T04:33:21.107267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:12:35.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593206+0000","last_change":"2024-05-17T04:31:39.896901+0000","last_active":"2024-05-17T04:33:21.593206+0000","last_peered":"2024-05-17T04:33:21.593206+0000","last_clean":"2024-05-17T04:33:21.593206+0000","last_became_active":"2024-05-17T04:31:39.896580+0000","last_became_peered":"2024-05-17T04:31:39.896580+0000","last_unstale":"2024-05-17T04:33:21.593206+0000","last_undegraded":"2024-05-17T04:33:21.593206+0000","last_fullsized":"2024-05-17T04:33:21.593206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:26:24.317276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.695913+0000","last_change":"2024-05-17T04:31:42.935437+0000","last_active":"2024-05-17T04:34:18.695913+0000","last_peered":"2024-05-17T04:34:18.695913+0000","last_clean":"2024-05-17T04:34:18.695913+0000","last_became_active":"2024-05-17T04:31:42.935067+0000","last_became_peered":"2024-05-17T04:31:42.935067+0000","last_unstale":"2024-05-17T04:34:18.695913+0000","last_undegraded":"2024-05-17T04:34:18.695913+0000","last_fullsized":"2024-05-17T04:34:18.695913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:56:20.766093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.110306+0000","last_change":"2024-05-17T04:33:21.109697+0000","last_active":"2024-05-17T04:33:21.110306+0000","last_peered":"2024-05-17T04:33:21.110306+0000","last_clean":"2024-05-17T04:33:21.110306+0000","last_became_active":"2024-05-17T04:33:21.109108+0000","last_became_peered":"2024-05-17T04:33:21.109108+0000","last_unstale":"2024-05-17T04:33:21.110306+0000","last_undegraded":"2024-05-17T04:33:21.110306+0000","last_fullsized":"2024-05-17T04:33:21.110306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:21:35.475395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.100892+0000","last_change":"2024-05-17T04:31:39.893877+0000","last_active":"2024-05-17T04:33:21.100892+0000","last_peered":"2024-05-17T04:33:21.100892+0000","last_clean":"2024-05-17T04:33:21.100892+0000","last_became_active":"2024-05-17T04:31:39.893479+0000","last_became_peered":"2024-05-17T04:31:39.893479+0000","last_unstale":"2024-05-17T04:33:21.100892+0000","last_undegraded":"2024-05-17T04:33:21.100892+0000","last_fullsized":"2024-05-17T04:33:21.100892+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:46:21.163132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.708977+0000","last_change":"2024-05-17T04:31:42.934449+0000","last_active":"2024-05-17T04:34:18.708977+0000","last_peered":"2024-05-17T04:34:18.708977+0000","last_clean":"2024-05-17T04:34:18.708977+0000","last_became_active":"2024-05-17T04:31:42.934054+0000","last_became_peered":"2024-05-17T04:31:42.934054+0000","last_unstale":"2024-05-17T04:34:18.708977+0000","last_undegraded":"2024-05-17T04:34:18.708977+0000","last_fullsized":"2024-05-17T04:34:18.708977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:57:35.344333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.936160+0000","last_change":"2024-05-17T04:31:55.986935+0000","last_active":"2024-05-17T04:33:20.936160+0000","last_peered":"2024-05-17T04:33:20.936160+0000","last_clean":"2024-05-17T04:33:20.936160+0000","last_became_active":"2024-05-17T04:31:55.986471+0000","last_became_peered":"2024-05-17T04:31:55.986471+0000","last_unstale":"2024-05-17T04:33:20.936160+0000","last_undegraded":"2024-05-17T04:33:20.936160+0000","last_fullsized":"2024-05-17T04:33:20.936160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:04:09.479135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.933187+0000","last_change":"2024-05-17T04:31:39.890000+0000","last_active":"2024-05-17T04:33:20.933187+0000","last_peered":"2024-05-17T04:33:20.933187+0000","last_clean":"2024-05-17T04:33:20.933187+0000","last_became_active":"2024-05-17T04:31:39.887979+0000","last_became_peered":"2024-05-17T04:31:39.887979+0000","last_unstale":"2024-05-17T04:33:20.933187+0000","last_undegraded":"2024-05-17T04:33:20.933187+0000","last_fullsized":"2024-05-17T04:33:20.933187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:44:13.580387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719173+0000","last_change":"2024-05-17T04:31:42.943980+0000","last_active":"2024-05-17T04:34:18.719173+0000","last_peered":"2024-05-17T04:34:18.719173+0000","last_clean":"2024-05-17T04:34:18.719173+0000","last_became_active":"2024-05-17T04:31:42.943647+0000","last_became_peered":"2024-05-17T04:31:42.943647+0000","last_unstale":"2024-05-17T04:34:18.719173+0000","last_undegraded":"2024-05-17T04:34:18.719173+0000","last_fullsized":"2024-05-17T04:34:18.719173+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:17:15.909198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935511+0000","last_change":"2024-05-17T04:31:56.178896+0000","last_active":"2024-05-17T04:33:20.935511+0000","last_peered":"2024-05-17T04:33:20.935511+0000","last_clean":"2024-05-17T04:33:20.935511+0000","last_became_active":"2024-05-17T04:31:56.178298+0000","last_became_peered":"2024-05-17T04:31:56.178298+0000","last_unstale":"2024-05-17T04:33:20.935511+0000","last_undegraded":"2024-05-17T04:33:20.935511+0000","last_fullsized":"2024-05-17T04:33:20.935511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T05:30:32.759024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:20.935193+0000","last_change":"2024-05-17T04:31:39.894705+0000","last_active":"2024-05-17T04:33:20.935193+0000","last_peered":"2024-05-17T04:33:20.935193+0000","last_clean":"2024-05-17T04:33:20.935193+0000","last_became_active":"2024-05-17T04:31:39.894312+0000","last_became_peered":"2024-05-17T04:31:39.894312+0000","last_unstale":"2024-05-17T04:33:20.935193+0000","last_undegraded":"2024-05-17T04:33:20.935193+0000","last_fullsized":"2024-05-17T04:33:20.935193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:41:13.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725949+0000","last_change":"2024-05-17T04:31:42.941171+0000","last_active":"2024-05-17T04:34:18.725949+0000","last_peered":"2024-05-17T04:34:18.725949+0000","last_clean":"2024-05-17T04:34:18.725949+0000","last_became_active":"2024-05-17T04:31:42.940724+0000","last_became_peered":"2024-05-17T04:31:42.940724+0000","last_unstale":"2024-05-17T04:34:18.725949+0000","last_undegraded":"2024-05-17T04:34:18.725949+0000","last_fullsized":"2024-05-17T04:34:18.725949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:26:08.933000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593215+0000","last_change":"2024-05-17T04:31:55.992427+0000","last_active":"2024-05-17T04:33:21.593215+0000","last_peered":"2024-05-17T04:33:21.593215+0000","last_clean":"2024-05-17T04:33:21.593215+0000","last_became_active":"2024-05-17T04:31:55.991936+0000","last_became_peered":"2024-05-17T04:31:55.991936+0000","last_unstale":"2024-05-17T04:33:21.593215+0000","last_undegraded":"2024-05-17T04:33:21.593215+0000","last_fullsized":"2024-05-17T04:33:21.593215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:30:01.610347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593298+0000","last_change":"2024-05-17T04:32:20.062349+0000","last_active":"2024-05-17T04:33:21.593298+0000","last_peered":"2024-05-17T04:33:21.593298+0000","last_clean":"2024-05-17T04:33:21.593298+0000","last_became_active":"2024-05-17T04:32:20.061708+0000","last_became_peered":"2024-05-17T04:32:20.061708+0000","last_unstale":"2024-05-17T04:33:21.593298+0000","last_undegraded":"2024-05-17T04:33:21.593298+0000","last_fullsized":"2024-05-17T04:33:21.593298+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:51:02.946670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.725338+0000","last_change":"2024-05-17T04:31:42.931998+0000","last_active":"2024-05-17T04:34:18.725338+0000","last_peered":"2024-05-17T04:34:18.725338+0000","last_clean":"2024-05-17T04:34:18.725338+0000","last_became_active":"2024-05-17T04:31:42.929898+0000","last_became_peered":"2024-05-17T04:31:42.929898+0000","last_unstale":"2024-05-17T04:34:18.725338+0000","last_undegraded":"2024-05-17T04:34:18.725338+0000","last_fullsized":"2024-05-17T04:34:18.725338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:55:35.349637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.593250+0000","last_change":"2024-05-17T04:31:55.990093+0000","last_active":"2024-05-17T04:33:21.593250+0000","last_peered":"2024-05-17T04:33:21.593250+0000","last_clean":"2024-05-17T04:33:21.593250+0000","last_became_active":"2024-05-17T04:31:55.989665+0000","last_became_peered":"2024-05-17T04:31:55.989665+0000","last_unstale":"2024-05-17T04:33:21.593250+0000","last_undegraded":"2024-05-17T04:33:21.593250+0000","last_fullsized":"2024-05-17T04:33:21.593250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T08:37:43.727322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.108607+0000","last_change":"2024-05-17T04:31:39.893601+0000","last_active":"2024-05-17T04:33:21.108607+0000","last_peered":"2024-05-17T04:33:21.108607+0000","last_clean":"2024-05-17T04:33:21.108607+0000","last_became_active":"2024-05-17T04:31:39.893141+0000","last_became_peered":"2024-05-17T04:31:39.893141+0000","last_unstale":"2024-05-17T04:33:21.108607+0000","last_undegraded":"2024-05-17T04:33:21.108607+0000","last_fullsized":"2024-05-17T04:33:21.108607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:05:09.173607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.723513+0000","last_change":"2024-05-17T04:31:42.932083+0000","last_active":"2024-05-17T04:34:18.723513+0000","last_peered":"2024-05-17T04:34:18.723513+0000","last_clean":"2024-05-17T04:34:18.723513+0000","last_became_active":"2024-05-17T04:31:42.931520+0000","last_became_peered":"2024-05-17T04:31:42.931520+0000","last_unstale":"2024-05-17T04:34:18.723513+0000","last_undegraded":"2024-05-17T04:34:18.723513+0000","last_fullsized":"2024-05-17T04:34:18.723513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T07:47:55.057982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.107506+0000","last_change":"2024-05-17T04:31:55.990315+0000","last_active":"2024-05-17T04:33:21.107506+0000","last_peered":"2024-05-17T04:33:21.107506+0000","last_clean":"2024-05-17T04:33:21.107506+0000","last_became_active":"2024-05-17T04:31:55.989565+0000","last_became_peered":"2024-05-17T04:31:55.989565+0000","last_unstale":"2024-05-17T04:33:21.107506+0000","last_undegraded":"2024-05-17T04:33:21.107506+0000","last_fullsized":"2024-05-17T04:33:21.107506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:54.961366+0000","last_clean_scrub_stamp":"2024-05-17T04:31:54.961366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:52:03.699211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:33:21.106305+0000","last_change":"2024-05-17T04:31:39.890621+0000","last_active":"2024-05-17T04:33:21.106305+0000","last_peered":"2024-05-17T04:33:21.106305+0000","last_clean":"2024-05-17T04:33:21.106305+0000","last_became_active":"2024-05-17T04:31:39.890180+0000","last_became_peered":"2024-05-17T04:31:39.890180+0000","last_unstale":"2024-05-17T04:33:21.106305+0000","last_undegraded":"2024-05-17T04:33:21.106305+0000","last_fullsized":"2024-05-17T04:33:21.106305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:38.873902+0000","last_clean_scrub_stamp":"2024-05-17T04:31:38.873902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:42.018482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.718579+0000","last_change":"2024-05-17T04:31:42.928951+0000","last_active":"2024-05-17T04:34:18.718579+0000","last_peered":"2024-05-17T04:34:18.718579+0000","last_clean":"2024-05-17T04:34:18.718579+0000","last_became_active":"2024-05-17T04:31:42.928382+0000","last_became_peered":"2024-05-17T04:31:42.928382+0000","last_unstale":"2024-05-17T04:34:18.718579+0000","last_undegraded":"2024-05-17T04:34:18.718579+0000","last_fullsized":"2024-05-17T04:34:18.718579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:17:36.307351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714846+0000","last_change":"2024-05-17T04:31:42.926377+0000","last_active":"2024-05-17T04:34:18.714846+0000","last_peered":"2024-05-17T04:34:18.714846+0000","last_clean":"2024-05-17T04:34:18.714846+0000","last_became_active":"2024-05-17T04:31:42.926007+0000","last_became_peered":"2024-05-17T04:31:42.926007+0000","last_unstale":"2024-05-17T04:34:18.714846+0000","last_undegraded":"2024-05-17T04:34:18.714846+0000","last_fullsized":"2024-05-17T04:34:18.714846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:43:45.456144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.719776+0000","last_change":"2024-05-17T04:31:42.932742+0000","last_active":"2024-05-17T04:34:18.719776+0000","last_peered":"2024-05-17T04:34:18.719776+0000","last_clean":"2024-05-17T04:34:18.719776+0000","last_became_active":"2024-05-17T04:31:42.932374+0000","last_became_peered":"2024-05-17T04:31:42.932374+0000","last_unstale":"2024-05-17T04:34:18.719776+0000","last_undegraded":"2024-05-17T04:34:18.719776+0000","last_fullsized":"2024-05-17T04:34:18.719776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T10:40:37.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713029+0000","last_change":"2024-05-17T04:31:42.940870+0000","last_active":"2024-05-17T04:34:18.713029+0000","last_peered":"2024-05-17T04:34:18.713029+0000","last_clean":"2024-05-17T04:34:18.713029+0000","last_became_active":"2024-05-17T04:31:42.940463+0000","last_became_peered":"2024-05-17T04:31:42.940463+0000","last_unstale":"2024-05-17T04:34:18.713029+0000","last_undegraded":"2024-05-17T04:34:18.713029+0000","last_fullsized":"2024-05-17T04:34:18.713029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:48:52.684601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.720264+0000","last_change":"2024-05-17T04:31:42.936055+0000","last_active":"2024-05-17T04:34:18.720264+0000","last_peered":"2024-05-17T04:34:18.720264+0000","last_clean":"2024-05-17T04:34:18.720264+0000","last_became_active":"2024-05-17T04:31:42.934013+0000","last_became_peered":"2024-05-17T04:31:42.934013+0000","last_unstale":"2024-05-17T04:34:18.720264+0000","last_undegraded":"2024-05-17T04:34:18.720264+0000","last_fullsized":"2024-05-17T04:34:18.720264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T13:03:10.502727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.709557+0000","last_change":"2024-05-17T04:31:42.942930+0000","last_active":"2024-05-17T04:34:18.709557+0000","last_peered":"2024-05-17T04:34:18.709557+0000","last_clean":"2024-05-17T04:34:18.709557+0000","last_became_active":"2024-05-17T04:31:42.942585+0000","last_became_peered":"2024-05-17T04:31:42.942585+0000","last_unstale":"2024-05-17T04:34:18.709557+0000","last_undegraded":"2024-05-17T04:34:18.709557+0000","last_fullsized":"2024-05-17T04:34:18.709557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T16:28:45.780505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.714651+0000","last_change":"2024-05-17T04:31:42.923752+0000","last_active":"2024-05-17T04:34:18.714651+0000","last_peered":"2024-05-17T04:34:18.714651+0000","last_clean":"2024-05-17T04:34:18.714651+0000","last_became_active":"2024-05-17T04:31:42.923375+0000","last_became_peered":"2024-05-17T04:31:42.923375+0000","last_unstale":"2024-05-17T04:34:18.714651+0000","last_undegraded":"2024-05-17T04:34:18.714651+0000","last_fullsized":"2024-05-17T04:34:18.714651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:32:13.755314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.731053+0000","last_change":"2024-05-17T04:31:42.928666+0000","last_active":"2024-05-17T04:34:18.731053+0000","last_peered":"2024-05-17T04:34:18.731053+0000","last_clean":"2024-05-17T04:34:18.731053+0000","last_became_active":"2024-05-17T04:31:42.928291+0000","last_became_peered":"2024-05-17T04:31:42.928291+0000","last_unstale":"2024-05-17T04:34:18.731053+0000","last_undegraded":"2024-05-17T04:34:18.731053+0000","last_fullsized":"2024-05-17T04:34:18.731053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:12:32.253374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.707560+0000","last_change":"2024-05-17T04:31:42.926316+0000","last_active":"2024-05-17T04:34:18.707560+0000","last_peered":"2024-05-17T04:34:18.707560+0000","last_clean":"2024-05-17T04:34:18.707560+0000","last_became_active":"2024-05-17T04:31:42.925823+0000","last_became_peered":"2024-05-17T04:31:42.925823+0000","last_unstale":"2024-05-17T04:34:18.707560+0000","last_undegraded":"2024-05-17T04:34:18.707560+0000","last_fullsized":"2024-05-17T04:34:18.707560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:16:15.670274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717555+0000","last_change":"2024-05-17T04:31:42.945587+0000","last_active":"2024-05-17T04:34:18.717555+0000","last_peered":"2024-05-17T04:34:18.717555+0000","last_clean":"2024-05-17T04:34:18.717555+0000","last_became_active":"2024-05-17T04:31:42.945238+0000","last_became_peered":"2024-05-17T04:31:42.945238+0000","last_unstale":"2024-05-17T04:34:18.717555+0000","last_undegraded":"2024-05-17T04:34:18.717555+0000","last_fullsized":"2024-05-17T04:34:18.717555+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T15:17:29.584023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.717028+0000","last_change":"2024-05-17T04:31:42.925883+0000","last_active":"2024-05-17T04:34:18.717028+0000","last_peered":"2024-05-17T04:34:18.717028+0000","last_clean":"2024-05-17T04:34:18.717028+0000","last_became_active":"2024-05-17T04:31:42.925506+0000","last_became_peered":"2024-05-17T04:31:42.925506+0000","last_unstale":"2024-05-17T04:34:18.717028+0000","last_undegraded":"2024-05-17T04:34:18.717028+0000","last_fullsized":"2024-05-17T04:34:18.717028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T14:31:16.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.730488+0000","last_change":"2024-05-17T04:31:42.928444+0000","last_active":"2024-05-17T04:34:18.730488+0000","last_peered":"2024-05-17T04:34:18.730488+0000","last_clean":"2024-05-17T04:34:18.730488+0000","last_became_active":"2024-05-17T04:31:42.928060+0000","last_became_peered":"2024-05-17T04:31:42.928060+0000","last_unstale":"2024-05-17T04:34:18.730488+0000","last_undegraded":"2024-05-17T04:34:18.730488+0000","last_fullsized":"2024-05-17T04:34:18.730488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T04:47:29.714401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.711617+0000","last_change":"2024-05-17T04:31:42.937010+0000","last_active":"2024-05-17T04:34:18.711617+0000","last_peered":"2024-05-17T04:34:18.711617+0000","last_clean":"2024-05-17T04:34:18.711617+0000","last_became_active":"2024-05-17T04:31:42.936675+0000","last_became_peered":"2024-05-17T04:31:42.936675+0000","last_unstale":"2024-05-17T04:34:18.711617+0000","last_undegraded":"2024-05-17T04:34:18.711617+0000","last_fullsized":"2024-05-17T04:34:18.711617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T06:06:22.421966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-17T04:34:18.713643+0000","last_change":"2024-05-17T04:31:42.924943+0000","last_active":"2024-05-17T04:34:18.713643+0000","last_peered":"2024-05-17T04:34:18.713643+0000","last_clean":"2024-05-17T04:34:18.713643+0000","last_became_active":"2024-05-17T04:31:42.924578+0000","last_became_peered":"2024-05-17T04:31:42.924578+0000","last_unstale":"2024-05-17T04:34:18.713643+0000","last_undegraded":"2024-05-17T04:34:18.713643+0000","last_fullsized":"2024-05-17T04:34:18.713643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T04:31:41.892505+0000","last_clean_scrub_stamp":"2024-05-17T04:31:41.892505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T12:22:58.614038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284318,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66995,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":687,"num_read_kb":414,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":515050,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1165127,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":519338,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":515118,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":1111744,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":510418,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":573231,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":511806,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":573949,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":510612,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":515600,"data_compressed":32304,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822740,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1106800,"data_compressed":68452,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34526,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36148,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T04:34:42.498 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-17T04:34:42.499 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-17T04:34:42.499 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-17T04:34:42.499 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-17T04:34:42.758 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:43.161 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:43 smithi050 ceph-mon[31739]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:43.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:43 smithi110 ceph-mon[35497]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:43.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:43 smithi105 ceph-mon[34795]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-17T04:34:44.292 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:44 smithi050 ceph-mon[31739]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:44 smithi110 ceph-mon[35497]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:44 smithi105 ceph-mon[34795]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 -- 172.21.15.50:0/2231017186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc354102e20 msgr2=0x7fc354103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 --2- 172.21.15.50:0/2231017186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc354102e20 0x7fc354103230 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc344009990 tx=0x7fc344009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 -- 172.21.15.50:0/2231017186 shutdown_connections 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 --2- 172.21.15.50:0/2231017186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3541080a0 0x7fc35410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 --2- 172.21.15.50:0/2231017186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc354102e20 0x7fc354103230 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 --2- 172.21.15.50:0/2231017186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc354107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 -- 172.21.15.50:0/2231017186 >> 172.21.15.50:0/2231017186 conn(0x7fc354079bd0 msgr2=0x7fc354079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 -- 172.21.15.50:0/2231017186 shutdown_connections 2024-05-17T04:34:44.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.572+0000 7fc358acd700 1 -- 172.21.15.50:0/2231017186 wait complete. 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.573+0000 7fc358acd700 1 Processor -- start 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.573+0000 7fc358acd700 1 -- start start 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc354102e20 0x7fc3541a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc3541a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc354112b80 con 0x7fc3541080a0 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc354112a00 con 0x7fc354107750 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc358acd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc354112d00 con 0x7fc354102e20 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40676/0 (socket says 172.21.15.50:40676) 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 -- 172.21.15.50:0/726982988 learned_addr learned my addr 172.21.15.50:0/726982988 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc35259c700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc354102e20 0x7fc3541a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:44.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc351d9b700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc3541a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 -- 172.21.15.50:0/726982988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc354102e20 msgr2=0x7fc3541a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc354102e20 0x7fc3541a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 -- 172.21.15.50:0/726982988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 msgr2=0x7fc3541a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc3541a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 -- 172.21.15.50:0/726982988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35419ce90 con 0x7fc3541080a0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc351d9b700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc3541a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.574+0000 7fc352d9d700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ada0 tx=0x7fc34c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.575+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c017020 con 0x7fc3541080a0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.575+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc34c00ce10 con 0x7fc3541080a0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.575+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35419d180 con 0x7fc3541080a0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.575+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c005280 con 0x7fc3541080a0 2024-05-17T04:34:44.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.575+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3541a9380 con 0x7fc3541080a0 2024-05-17T04:34:44.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.576+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3541042d0 con 0x7fc3541080a0 2024-05-17T04:34:44.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.577+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc34c00c980 con 0x7fc3541080a0 2024-05-17T04:34:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.578+0000 7fc3437fe700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 0x7fc33c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.578+0000 7fc35259c700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 0x7fc33c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:44.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.578+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc34c0e2230 con 0x7fc3541080a0 2024-05-17T04:34:44.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.579+0000 7fc35259c700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 0x7fc33c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b5c0 tx=0x7fc344005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:44.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.581+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc34c021020 con 0x7fc3541080a0 2024-05-17T04:34:44.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.843+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc35419dc70 con 0x7fc3541080a0 2024-05-17T04:34:44.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.844+0000 7fc3437fe700 1 -- 172.21.15.50:0/726982988 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc34c0a9c50 con 0x7fc3541080a0 2024-05-17T04:34:44.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:44.844 INFO:teuthology.orchestra.run.smithi050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T04:34:44.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 msgr2=0x7fc33c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:44.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 0x7fc33c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b5c0 tx=0x7fc344005960 comp rx=0 tx=0).stop 2024-05-17T04:34:44.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 msgr2=0x7fc35419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00ada0 tx=0x7fc34c00c330 comp rx=0 tx=0).stop 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 shutdown_connections 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc33c0b2880 0x7fc33c0b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3541080a0 0x7fc35419c760 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc354102e20 0x7fc3541a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 --2- 172.21.15.50:0/726982988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc354107750 0x7fc3541a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 >> 172.21.15.50:0/726982988 conn(0x7fc354079bd0 msgr2=0x7fc35410eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 shutdown_connections 2024-05-17T04:34:44.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:44.846+0000 7fc358acd700 1 -- 172.21.15.50:0/726982988 wait complete. 2024-05-17T04:34:45.225 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:45 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/726982988' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T04:34:45.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:45 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/726982988' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T04:34:45.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:45 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/726982988' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-17T04:34:45.714 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-17T04:34:45.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:34:45.981 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:46.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:46 smithi050 ceph-mon[31739]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:46 smithi105 ceph-mon[34795]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:46 smithi110 ceph-mon[35497]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:47.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/1825081313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f8103f20 msgr2=0x7fc9f8104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/1825081313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f8103f20 0x7fc9f8104330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009990 tx=0x7fc9e8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/1825081313 shutdown_connections 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/1825081313 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f81091a0 0x7fc9f8110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/1825081313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f8103f20 0x7fc9f8104330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/1825081313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 0x7fc9f8108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/1825081313 >> 172.21.15.50:0/1825081313 conn(0x7fc9f80ff9c0 msgr2=0x7fc9f8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/1825081313 shutdown_connections 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/1825081313 wait complete. 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.822+0000 7fc9fd7ea700 1 Processor -- start 2024-05-17T04:34:47.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 -- start start 2024-05-17T04:34:47.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:47.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 0x7fc9f819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8113c80 con 0x7fc9f81091a0 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8113b00 con 0x7fc9f8108850 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9fd7ea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8113e00 con 0x7fc9f8103f20 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40690/0 (socket says 172.21.15.50:40690) 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:54714/0 (socket says 172.21.15.50:54714) 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f77fe700 1 -- 172.21.15.50:0/4258419493 learned_addr learned my addr 172.21.15.50:0/4258419493 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.823+0000 7fc9f67fc700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 0x7fc9f819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 -- 172.21.15.50:0/4258419493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 msgr2=0x7fc9f819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:47.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 -- 172.21.15.50:0/4258419493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 msgr2=0x7fc9f819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 0x7fc9f819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 -- 172.21.15.50:0/4258419493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f819f5b0 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f6ffd700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9f77fe700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007ee0 tx=0x7fc9f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0004bb0 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9f0004d50 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f819f8a0 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00058b0 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.824+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f81a71c0 con 0x7fc9f81091a0 2024-05-17T04:34:47.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.825+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f8104330 con 0x7fc9f81091a0 2024-05-17T04:34:47.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.828+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc9f001a020 con 0x7fc9f81091a0 2024-05-17T04:34:47.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.828+0000 7fc9dffff700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 0x7fc9e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:47.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.829+0000 7fc9f6ffd700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 0x7fc9e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:47.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.829+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc9f00e2240 con 0x7fc9f81091a0 2024-05-17T04:34:47.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.829+0000 7fc9f6ffd700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 0x7fc9e00b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800b5c0 tx=0x7fc9e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:47.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:47.832+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9f0010070 con 0x7fc9f81091a0 2024-05-17T04:34:48.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.095+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9f802ea80 con 0x7fc9f81091a0 2024-05-17T04:34:48.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.096+0000 7fc9dffff700 1 -- 172.21.15.50:0/4258419493 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7fc9f00a9bd0 con 0x7fc9f81091a0 2024-05-17T04:34:48.096 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:48.096 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 msgr2=0x7fc9e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 0x7fc9e00b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800b5c0 tx=0x7fc9e8005960 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 msgr2=0x7fc9f819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007ee0 tx=0x7fc9f000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 shutdown_connections 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc9e00b2880 0x7fc9e00b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9f81091a0 0x7fc9f819ef10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9f8103f20 0x7fc9f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 --2- 172.21.15.50:0/4258419493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9f8108850 0x7fc9f819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.098+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 >> 172.21.15.50:0/4258419493 conn(0x7fc9f80ff9c0 msgr2=0x7fc9f8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.099+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 shutdown_connections 2024-05-17T04:34:48.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:48.099+0000 7fc9fd7ea700 1 -- 172.21.15.50:0/4258419493 wait complete. 2024-05-17T04:34:48.100 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:34:48.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:48 smithi050 ceph-mon[31739]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:48 smithi105 ceph-mon[34795]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:48 smithi110 ceph-mon[35497]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:48.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:34:49.101 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:49.370 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/4258419493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:49.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/4258419493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:49.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/4258419493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:50 smithi050 ceph-mon[31739]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:50 smithi105 ceph-mon[34795]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:50 smithi110 ceph-mon[35497]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 -- 172.21.15.50:0/3463647923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c1080a0 msgr2=0x7fc33c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 --2- 172.21.15.50:0/3463647923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c1080a0 0x7fc33c10fa00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b0a0 tx=0x7fc33400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 -- 172.21.15.50:0/3463647923 shutdown_connections 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 --2- 172.21.15.50:0/3463647923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c1080a0 0x7fc33c10fa00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 --2- 172.21.15.50:0/3463647923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c102e70 0x7fc33c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 --2- 172.21.15.50:0/3463647923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 0x7fc33c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 -- 172.21.15.50:0/3463647923 >> 172.21.15.50:0/3463647923 conn(0x7fc33c079ae0 msgr2=0x7fc33c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 -- 172.21.15.50:0/3463647923 shutdown_connections 2024-05-17T04:34:50.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.958+0000 7fc340d81700 1 -- 172.21.15.50:0/3463647923 wait complete. 2024-05-17T04:34:50.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.959+0000 7fc340d81700 1 Processor -- start 2024-05-17T04:34:50.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.959+0000 7fc340d81700 1 -- start start 2024-05-17T04:34:50.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 0x7fc33c1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c1080a0 0x7fc33c105c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c112b90 con 0x7fc33c102e70 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c112a10 con 0x7fc33c107750 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc340d81700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c112d10 con 0x7fc33c1080a0 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44262/0 (socket says 172.21.15.50:44262) 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 -- 172.21.15.50:0/3353846928 learned_addr learned my addr 172.21.15.50:0/3353846928 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33ad9d700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c1080a0 0x7fc33c105c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc339d9b700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 0x7fc33c1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 -- 172.21.15.50:0/3353846928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c1080a0 msgr2=0x7fc33c105c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:50.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c1080a0 0x7fc33c105c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 -- 172.21.15.50:0/3353846928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 msgr2=0x7fc33c1056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 0x7fc33c1056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.960+0000 7fc33a59c700 1 -- 172.21.15.50:0/3353846928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c1a8a30 con 0x7fc33c102e70 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc33a59c700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00d950 tx=0x7fc32c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c016070 con 0x7fc33c102e70 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c1a8cc0 con 0x7fc33c102e70 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc32c00fd20 con 0x7fc33c102e70 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00e3b0 con 0x7fc33c102e70 2024-05-17T04:34:50.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.961+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c1a91b0 con 0x7fc33c102e70 2024-05-17T04:34:50.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.962+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c103280 con 0x7fc33c102e70 2024-05-17T04:34:50.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.963+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc32c019070 con 0x7fc33c102e70 2024-05-17T04:34:50.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.965+0000 7fc32b7fe700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 0x7fc3240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:50.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.965+0000 7fc339d9b700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 0x7fc3240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:50.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.965+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc32c0e2000 con 0x7fc33c102e70 2024-05-17T04:34:50.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.965+0000 7fc339d9b700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 0x7fc3240b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc33c104af0 tx=0x7fc33000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:50.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:50.968+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc32c0a9990 con 0x7fc33c102e70 2024-05-17T04:34:51.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.234+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc33c1066e0 con 0x7fc33c102e70 2024-05-17T04:34:51.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.234+0000 7fc32b7fe700 1 -- 172.21.15.50:0/3353846928 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7fc32c0a9290 con 0x7fc33c102e70 2024-05-17T04:34:51.236 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 msgr2=0x7fc3240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 0x7fc3240b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc33c104af0 tx=0x7fc33000a380 comp rx=0 tx=0).stop 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 msgr2=0x7fc33c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00d950 tx=0x7fc32c00dc60 comp rx=0 tx=0).stop 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 shutdown_connections 2024-05-17T04:34:51.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc33c1080a0 0x7fc33c105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc3240b2880 0x7fc3240b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc33c102e70 0x7fc33c1051b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 --2- 172.21.15.50:0/3353846928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc33c107750 0x7fc33c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 >> 172.21.15.50:0/3353846928 conn(0x7fc33c079ae0 msgr2=0x7fc33c06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 shutdown_connections 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:51.237+0000 7fc340d81700 1 -- 172.21.15.50:0/3353846928 wait complete. 2024-05-17T04:34:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:34:51.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:34:52.229 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:52 smithi050 ceph-mon[31739]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:52 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3353846928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:52 smithi105 ceph-mon[34795]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:52.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:52 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3353846928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:52 smithi110 ceph-mon[35497]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:52 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3353846928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.028+0000 7fe4db255700 1 -- 172.21.15.50:0/2084102178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4107830 msgr2=0x7fe4d410f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.028+0000 7fe4db255700 1 --2- 172.21.15.50:0/2084102178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4107830 0x7fe4d410f190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00b0a0 tx=0x7fe4cc00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 -- 172.21.15.50:0/2084102178 shutdown_connections 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 --2- 172.21.15.50:0/2084102178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4107830 0x7fe4d410f190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 --2- 172.21.15.50:0/2084102178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d41025b0 0x7fe4d41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 --2- 172.21.15.50:0/2084102178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 -- 172.21.15.50:0/2084102178 >> 172.21.15.50:0/2084102178 conn(0x7fe4d40780d0 msgr2=0x7fe4d4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 -- 172.21.15.50:0/2084102178 shutdown_connections 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 -- 172.21.15.50:0/2084102178 wait complete. 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.029+0000 7fe4db255700 1 Processor -- start 2024-05-17T04:34:54.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 -- start start 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4107830 0x7fe4d4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112ba0 con 0x7fe4d4106ee0 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112a20 con 0x7fe4d41025b0 2024-05-17T04:34:54.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4db255700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4112d20 con 0x7fe4d4107830 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d8ff1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d8ff1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36026/0 (socket says 172.21.15.50:36026) 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d8ff1700 1 -- 172.21.15.50:0/823349410 learned_addr learned my addr 172.21.15.50:0/823349410 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d97f2700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4107830 0x7fe4d4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 -- 172.21.15.50:0/823349410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4107830 msgr2=0x7fe4d4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:54.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4107830 0x7fe4d4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 -- 172.21.15.50:0/823349410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 msgr2=0x7fe4d41a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d3fff700 1 -- 172.21.15.50:0/823349410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d4199160 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.030+0000 7fe4d8ff1700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4d3fff700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41983d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4075c90 tx=0x7fe4c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8016070 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c800fd20 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d41b5910 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800e3b0 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.031+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d41b5e20 con 0x7fe4d4106ee0 2024-05-17T04:34:54.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.032+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d41029c0 con 0x7fe4d4106ee0 2024-05-17T04:34:54.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.034+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe4c8019070 con 0x7fe4d4106ee0 2024-05-17T04:34:54.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.035+0000 7fe4d1ffb700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 0x7fe4bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:54.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.035+0000 7fe4d8ff1700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 0x7fe4bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:54.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.035+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4c80e23c0 con 0x7fe4d4106ee0 2024-05-17T04:34:54.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.035+0000 7fe4d8ff1700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 0x7fe4bc0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4011550 tx=0x7fe4c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:54.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.038+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4c80a9d50 con 0x7fe4d4106ee0 2024-05-17T04:34:54.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.256+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d41b64d0 con 0x7fe4d4106ee0 2024-05-17T04:34:54.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.257+0000 7fe4d1ffb700 1 -- 172.21.15.50:0/823349410 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7fe4c80a9650 con 0x7fe4d4106ee0 2024-05-17T04:34:54.259 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:54.259 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 msgr2=0x7fe4bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 0x7fe4bc0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4011550 tx=0x7fe4c4009260 comp rx=0 tx=0).stop 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 msgr2=0x7fe4d41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41983d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4075c90 tx=0x7fe4c800dc30 comp rx=0 tx=0).stop 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 shutdown_connections 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4107830 0x7fe4d4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d41025b0 0x7fe4d41a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4bc0b2880 0x7fe4bc0b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 --2- 172.21.15.50:0/823349410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d4106ee0 0x7fe4d41983d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 >> 172.21.15.50:0/823349410 conn(0x7fe4d40780d0 msgr2=0x7fe4d4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 shutdown_connections 2024-05-17T04:34:54.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:54.260+0000 7fe4db255700 1 -- 172.21.15.50:0/823349410 wait complete. 2024-05-17T04:34:54.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:54 smithi050 ceph-mon[31739]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:54 smithi105 ceph-mon[34795]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:54 smithi110 ceph-mon[35497]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:55.022 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:34:55.278 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:55 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/823349410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:34:55.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:55 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/823349410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:34:55.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:55 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/823349410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:34:56.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:56 smithi050 ceph-mon[31739]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:56 smithi105 ceph-mon[34795]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:56 smithi110 ceph-mon[35497]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:34:57.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 -- 172.21.15.50:0/1112560013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc1105e0 msgr2=0x7f6cdc112a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 --2- 172.21.15.50:0/1112560013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc112a00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400b0a0 tx=0x7f6cd400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 -- 172.21.15.50:0/1112560013 shutdown_connections 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 --2- 172.21.15.50:0/1112560013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc112a00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 --2- 172.21.15.50:0/1112560013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc102a30 0x7f6cdc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 --2- 172.21.15.50:0/1112560013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc103380 0x7f6cdc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 -- 172.21.15.50:0/1112560013 >> 172.21.15.50:0/1112560013 conn(0x7f6cdc079ae0 msgr2=0x7f6cdc079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 -- 172.21.15.50:0/1112560013 shutdown_connections 2024-05-17T04:34:57.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.032+0000 7f6ce3780700 1 -- 172.21.15.50:0/1112560013 wait complete. 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.033+0000 7f6ce3780700 1 Processor -- start 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.033+0000 7f6ce3780700 1 -- start start 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 0x7f6cdc1a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc115ee0 con 0x7f6cdc102a30 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc115d60 con 0x7f6cdc103380 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce3780700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc116060 con 0x7f6cdc1105e0 2024-05-17T04:34:57.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44300/0 (socket says 172.21.15.50:44300) 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 -- 172.21.15.50:0/657449428 learned_addr learned my addr 172.21.15.50:0/657449428 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce1d1d700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce0d1b700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 0x7f6cdc1a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 -- 172.21.15.50:0/657449428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 msgr2=0x7f6cdc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 -- 172.21.15.50:0/657449428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 msgr2=0x7f6cdc1a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 0x7f6cdc1a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 -- 172.21.15.50:0/657449428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc19ce20 con 0x7f6cdc102a30 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce1d1d700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce0d1b700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 0x7f6cdc1a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.034+0000 7f6ce151c700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d950 tx=0x7f6cd800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.035+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8016070 con 0x7f6cdc102a30 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.035+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc19d110 con 0x7f6cdc102a30 2024-05-17T04:34:57.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.035+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cd8010460 con 0x7f6cdc102a30 2024-05-17T04:34:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.035+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800f620 con 0x7f6cdc102a30 2024-05-17T04:34:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.035+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a9310 con 0x7f6cdc102a30 2024-05-17T04:34:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.036+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc06c8d0 con 0x7f6cdc102a30 2024-05-17T04:34:57.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.037+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6cd8019070 con 0x7f6cdc102a30 2024-05-17T04:34:57.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.038+0000 7f6cd27fc700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 0x7f6cc80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:34:57.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.038+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6cd80e2230 con 0x7f6cdc102a30 2024-05-17T04:34:57.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.038+0000 7f6ce0d1b700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 0x7f6cc80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:34:57.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.039+0000 7f6ce0d1b700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 0x7f6cc80b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc19de90 tx=0x7f6ccc005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:34:57.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.041+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6cd80a9b60 con 0x7f6cdc102a30 2024-05-17T04:34:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.302+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cdc102e40 con 0x7f6cdc102a30 2024-05-17T04:34:57.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.302+0000 7f6cd27fc700 1 -- 172.21.15.50:0/657449428 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7f6cd80a9460 con 0x7f6cdc102a30 2024-05-17T04:34:57.303 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:34:57.304 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:34:57.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 msgr2=0x7f6cc80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:57.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 0x7f6cc80b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc19de90 tx=0x7f6ccc005d80 comp rx=0 tx=0).stop 2024-05-17T04:34:57.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 msgr2=0x7f6cdc1a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:34:57.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800d950 tx=0x7f6cd800dd10 comp rx=0 tx=0).stop 2024-05-17T04:34:57.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 shutdown_connections 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc1105e0 0x7f6cdc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6cc80b2880 0x7f6cc80b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cdc102a30 0x7f6cdc1a2560 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 --2- 172.21.15.50:0/657449428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cdc103380 0x7f6cdc1a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 >> 172.21.15.50:0/657449428 conn(0x7f6cdc079ae0 msgr2=0x7f6cdc111a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 shutdown_connections 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:34:57.305+0000 7f6ce3780700 1 -- 172.21.15.50:0/657449428 wait complete. 2024-05-17T04:34:57.307 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:34:58.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T04:31:45.150094+0000', 'modified': '2024-05-17T04:32:53.059994+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24499, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/1830970686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1830970686}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1830970686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/1276100312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 1276100312}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1276100312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/1162023031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1162023031}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1162023031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.110:6837/2935546223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2935546223}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2935546223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.110:6839/2082916819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 2082916819}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 2082916819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/3722840784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 3722840784}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 3722840784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24487, 24506, 24499]} 2024-05-17T04:34:58.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T04:34:58.046 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-17T04:34:58.304 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:34:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:58 smithi050 ceph-mon[31739]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:34:58 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/657449428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:58 smithi105 ceph-mon[34795]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:58.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:34:58 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/657449428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:34:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:58 smithi110 ceph-mon[35497]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:34:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:34:58 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/657449428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 -- 172.21.15.50:0/487943573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24103f10 msgr2=0x7fef24104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 --2- 172.21.15.50:0/487943573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24103f10 0x7fef24104320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fef14009990 tx=0x7fef14009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 -- 172.21.15.50:0/487943573 shutdown_connections 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 --2- 172.21.15.50:0/487943573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 0x7fef24110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 --2- 172.21.15.50:0/487943573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24103f10 0x7fef24104320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 --2- 172.21.15.50:0/487943573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24108840 0x7fef24108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 -- 172.21.15.50:0/487943573 >> 172.21.15.50:0/487943573 conn(0x7fef240ff990 msgr2=0x7fef24101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 -- 172.21.15.50:0/487943573 shutdown_connections 2024-05-17T04:35:00.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.105+0000 7fef296bd700 1 -- 172.21.15.50:0/487943573 wait complete. 2024-05-17T04:35:00.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 Processor -- start 2024-05-17T04:35:00.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 -- start start 2024-05-17T04:35:00.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24103f10 0x7fef2419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 0x7fef2419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef24113c70 con 0x7fef24108840 2024-05-17T04:35:00.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fef24113af0 con 0x7fef24103f10 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef296bd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fef24113df0 con 0x7fef24109190 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55676/0 (socket says 172.21.15.50:55676) 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.106+0000 7fef227fc700 1 -- 172.21.15.50:0/2530702268 learned_addr learned my addr 172.21.15.50:0/2530702268 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef22ffd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24103f10 0x7fef2419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef237fe700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 0x7fef2419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 -- 172.21.15.50:0/2530702268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 msgr2=0x7fef2419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 0x7fef2419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 -- 172.21.15.50:0/2530702268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24103f10 msgr2=0x7fef2419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24103f10 0x7fef2419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 -- 172.21.15.50:0/2530702268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2419f5a0 con 0x7fef24108840 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef227fc700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef1800e970 tx=0x7fef1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef0bfff700 1 -- 172.21.15.50:0/2530702268 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18004a30 con 0x7fef24108840 2024-05-17T04:35:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef0bfff700 1 -- 172.21.15.50:0/2530702268 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef1800cc70 con 0x7fef24108840 2024-05-17T04:35:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2419f890 con 0x7fef24108840 2024-05-17T04:35:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef0bfff700 1 -- 172.21.15.50:0/2530702268 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18005520 con 0x7fef24108840 2024-05-17T04:35:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.107+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241a71b0 con 0x7fef24108840 2024-05-17T04:35:00.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.110+0000 7fef0bfff700 1 -- 172.21.15.50:0/2530702268 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef18021440 con 0x7fef24108840 2024-05-17T04:35:00.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.111+0000 7fef0bfff700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.111+0000 7fef22ffd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.111+0000 7fef0bfff700 1 -- 172.21.15.50:0/2530702268 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fef180e2770 con 0x7fef24108840 2024-05-17T04:35:00.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.111+0000 7fef38098b80 1 Processor -- start 2024-05-17T04:35:00.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 -- start start 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aaa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194aa5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561949d3240 con 0x556194aaaa50 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5561949c2f80 con 0x556194aa5120 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef38098b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5561949c6e20 con 0x5561949269a0 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef22ffd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 0x7fef0c0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b6f0 tx=0x7fef1400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55684/0 (socket says 172.21.15.50:55684) 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 -- 172.21.15.50:0/278761939 learned_addr learned my addr 172.21.15.50:0/278761939 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef22ffd700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aaa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef237fe700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194aa5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 -- 172.21.15.50:0/278761939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 msgr2=0x556194aaa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aaa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 -- 172.21.15.50:0/278761939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 msgr2=0x556194aa5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194aa5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef227fc700 1 -- 172.21.15.50:0/278761939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55619496c540 con 0x556194aaaa50 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.112+0000 7fef22ffd700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aaa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.113+0000 7fef227fc700 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef180a9930 tx=0x7fef180a9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aadc7ae836c5dbc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.113+0000 7fef097fa700 1 -- 172.21.15.50:0/278761939 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180c3070 con 0x556194aaaa50 2024-05-17T04:35:00.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.113+0000 7fef097fa700 1 -- 172.21.15.50:0/278761939 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef180b2150 con 0x556194aaaa50 2024-05-17T04:35:00.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.113+0000 7fef097fa700 1 -- 172.21.15.50:0/278761939 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1801fdd0 con 0x556194aaaa50 2024-05-17T04:35:00.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- 172.21.15.50:0/278761939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 msgr2=0x556194aa4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef180a9930 tx=0x7fef180a9210 comp rx=0 tx=0).stop 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- 172.21.15.50:0/278761939 shutdown_connections 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194aa5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aaa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 --2- 172.21.15.50:0/278761939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aa4be0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- 172.21.15.50:0/278761939 >> 172.21.15.50:0/278761939 conn(0x55619496aeb0 msgr2=0x556194a9fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- 172.21.15.50:0/278761939 shutdown_connections 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- 172.21.15.50:0/278761939 wait complete. 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 -- start start 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.114+0000 7fef38098b80 1 Processor -- start 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561949269a0 0x556194aae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 0x556194aae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561948ca100 con 0x556194aaaa50 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556194963e30 con 0x5561949269a0 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556194ab1340 con 0x556194aa5120 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef38098b80 10 client.0 _ll_register_callbacks cb 0x55619495dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef237fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 0x556194aae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef237fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55686/0 (socket says 172.21.15.50:55686) 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef237fe700 1 -- 172.21.15.50:0/1015445058 learned_addr learned my addr 172.21.15.50:0/1015445058 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561949269a0 0x556194aae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 -- 172.21.15.50:0/1015445058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561949269a0 msgr2=0x556194aae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561949269a0 0x556194aae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 -- 172.21.15.50:0/1015445058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 msgr2=0x556194aaec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef227fc700 1 -- 172.21.15.50:0/1015445058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556194913430 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.115+0000 7fef237fe700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef227fc700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 0x556194aae6f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x556194aa9fe0 tx=0x7fef180aa180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85b07e61f5eaa8c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180c3030 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fef180a5f90 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180b2150 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef180add50 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 10 client.24898 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556194abbdb0 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef180b2150 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fef180a9c00 con 0x556194aa5120 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 10 client.24898 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 10 client.24898 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T04:35:00.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 20 client.24898 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194abdef0 0x556194abf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 4 client.24898 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.116+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 --> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561949d5190 con 0x556194abdef0 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.117+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194abdef0 0x556194abf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.117+0000 7fef06ff5700 10 client.24898 ms_handle_connect on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:00.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.117+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194abdef0 0x556194abf360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c5044c24263c66b server_cookie=cf83c920ec076be3 in_seq=0 out_seq=0 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef06ff5700 1 -- 172.21.15.50:0/1015445058 <== mds.2 v2:172.21.15.110:6836/2935546223 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561949d5190 con 0x556194abdef0 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef06ff5700 10 client.24898 handle_command_reply: tid=0 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 client.24898 shutdown 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 msgr2=0x556194aae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 0x556194aae6f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x556194aa9fe0 tx=0x7fef180aa180 comp rx=0 tx=0).stop 2024-05-17T04:35:00.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 shutdown_connections 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aaaa50 0x556194aaec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194abdef0 0x556194abf360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561949269a0 0x556194aae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 --2- 172.21.15.50:0/1015445058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aa5120 0x556194aae6f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 >> 172.21.15.50:0/1015445058 conn(0x55619496aeb0 msgr2=0x556194a99e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef06ff5700 0 client.24898 ms_handle_reset on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 shutdown_connections 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 1 -- 172.21.15.50:0/1015445058 wait complete. 2024-05-17T04:35:00.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.118+0000 7fef38098b80 20 client.24898 trim_cache size 0 max 16384 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 Processor -- start 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 -- start start 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aa4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aaaa50 0x556194ab09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556194abd6a0 con 0x556194aa5120 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556194abdd70 con 0x556194aaaa50 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef38098b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556194abe440 con 0x5561949269a0 2024-05-17T04:35:00.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef227fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55692/0 (socket says 172.21.15.50:55692) 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef227fc700 1 -- 172.21.15.50:0/855933331 learned_addr learned my addr 172.21.15.50:0/855933331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.140+0000 7fef22ffd700 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aa4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef237fe700 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aaaa50 0x556194ab09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 -- 172.21.15.50:0/855933331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 msgr2=0x556194aa4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aa4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 -- 172.21.15.50:0/855933331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aaaa50 msgr2=0x556194ab09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aaaa50 0x556194ab09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 -- 172.21.15.50:0/855933331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556194a9bee0 con 0x556194aa5120 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef227fc700 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fef180adf10 tx=0x7fef180aa1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2322f03b09084965 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef08ff9700 1 -- 172.21.15.50:0/855933331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180b7070 con 0x556194aa5120 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef08ff9700 1 -- 172.21.15.50:0/855933331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef1800ce20 con 0x556194aa5120 2024-05-17T04:35:00.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.141+0000 7fef08ff9700 1 -- 172.21.15.50:0/855933331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180c3030 con 0x556194aa5120 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- 172.21.15.50:0/855933331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 msgr2=0x556194abd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fef180adf10 tx=0x7fef180aa1f0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- 172.21.15.50:0/855933331 shutdown_connections 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aaaa50 0x556194ab09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561949269a0 0x556194aa4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 --2- 172.21.15.50:0/855933331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556194aa5120 0x556194abd100 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- 172.21.15.50:0/855933331 >> 172.21.15.50:0/855933331 conn(0x55619496aeb0 msgr2=0x556194aa7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- 172.21.15.50:0/855933331 shutdown_connections 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- 172.21.15.50:0/855933331 wait complete. 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 -- start start 2024-05-17T04:35:00.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.142+0000 7fef38098b80 1 Processor -- start 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194ac3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aaaa50 0x556194ac42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556194abf1e0 con 0x5561949269a0 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556194abeb10 con 0x556194aa5120 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556194abf8b0 con 0x556194aaaa50 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef38098b80 10 client.0 _ll_register_callbacks cb 0x55619495dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55708/0 (socket says 172.21.15.50:55708) 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 -- 172.21.15.50:0/1748247310 learned_addr learned my addr 172.21.15.50:0/1748247310 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef237fe700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aaaa50 0x556194ac42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef227fc700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194ac3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 -- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aaaa50 msgr2=0x556194ac42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aaaa50 0x556194ac42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 -- 172.21.15.50:0/1748247310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 msgr2=0x556194ac3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194ac3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 -- 172.21.15.50:0/1748247310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556194ac4810 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef227fc700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194ac3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef22ffd700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fef14009990 tx=0x7fef1401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3008d98969e0f2d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.143+0000 7fef06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14036060 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fef14031910 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1800ce20 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef1403d050 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556194b28ff0 con 0x5561949269a0 2024-05-17T04:35:00.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef1800ce20 con 0x5561949269a0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fef14022070 con 0x5561949269a0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 10 client.15081 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 20 client.15081 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194b2b4a0 0x556194b2c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 4 client.15081 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.144+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 --> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] -- command(tid 0: {"prefix": "status"}) -- 0x5561949ce970 con 0x556194b2b4a0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef227fc700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194b2b4a0 0x556194b2c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef227fc700 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194b2b4a0 0x556194b2c910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5776926c7535392d server_cookie=58d1a6b374e96c17 in_seq=0 out_seq=0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef06ff5700 10 client.15081 ms_handle_connect on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef06ff5700 1 -- 172.21.15.50:0/1748247310 <== mds.2 v2:172.21.15.110:6836/2935546223 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5561949ce970 con 0x556194b2b4a0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef06ff5700 10 client.15081 handle_command_reply: tid=0 2024-05-17T04:35:00.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 client.15081 shutdown 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 msgr2=0x556194ac8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fef14009990 tx=0x7fef1401a050 comp rx=0 tx=0).stop 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 shutdown_connections 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556194aaaa50 0x556194ac42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561949269a0 0x556194ac8a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x556194b2b4a0 0x556194b2c910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 --2- 172.21.15.50:0/1748247310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556194aa5120 0x556194ac3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 >> 172.21.15.50:0/1748247310 conn(0x55619496aeb0 msgr2=0x556194aa3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.145+0000 7fef06ff5700 0 client.15081 ms_handle_reset on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 shutdown_connections 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef38098b80 1 -- 172.21.15.50:0/1748247310 wait complete. 2024-05-17T04:35:00.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef38098b80 20 client.15081 trim_cache size 0 max 16384 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24487, 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 132.11427003700001, 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:00.151 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 227.53992524500001 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 msgr2=0x7fef0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 0x7fef0c0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b6f0 tx=0x7fef1400b280 comp rx=0 tx=0).stop 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 msgr2=0x7fef2419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fef1800e970 tx=0x7fef1800ec80 comp rx=0 tx=0).stop 2024-05-17T04:35:00.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 shutdown_connections 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef24109190 0x7fef2419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fef24103f10 0x7fef2419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fef0c0b2880 0x7fef0c0b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 --2- 172.21.15.50:0/2530702268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef24108840 0x7fef2419ac70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 >> 172.21.15.50:0/2530702268 conn(0x7fef240ff990 msgr2=0x7fef24101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.146+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 shutdown_connections 2024-05-17T04:35:00.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:00.147+0000 7fef296bd700 1 -- 172.21.15.50:0/2530702268 wait complete. 2024-05-17T04:35:00.520 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:00 smithi050 ceph-mon[31739]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:00 smithi105 ceph-mon[34795]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:00 smithi110 ceph-mon[35497]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:00.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T04:35:01.242 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:02.369 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:02 smithi110 ceph-mon[35497]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:02.447 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:02 smithi050 ceph-mon[31739]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:02 smithi105 ceph-mon[34795]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 -- 172.21.15.50:0/3763632251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604102e00 msgr2=0x7ff604103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 --2- 172.21.15.50:0/3763632251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604102e00 0x7ff604103210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009990 tx=0x7ff5f4009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 -- 172.21.15.50:0/3763632251 shutdown_connections 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 --2- 172.21.15.50:0/3763632251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 0x7ff60410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 --2- 172.21.15.50:0/3763632251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604102e00 0x7ff604103210 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 --2- 172.21.15.50:0/3763632251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604107730 0x7ff604107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 -- 172.21.15.50:0/3763632251 >> 172.21.15.50:0/3763632251 conn(0x7ff604079bd0 msgr2=0x7ff604079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.059+0000 7ff608979700 1 -- 172.21.15.50:0/3763632251 shutdown_connections 2024-05-17T04:35:03.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.060+0000 7ff608979700 1 -- 172.21.15.50:0/3763632251 wait complete. 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 Processor -- start 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 -- start start 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 0x7ff60419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 0x7ff60419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff604112b60 con 0x7ff604107730 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041129e0 con 0x7ff604108080 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff608979700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff604112ce0 con 0x7ff604102e00 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff601d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff601d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55720/0 (socket says 172.21.15.50:55720) 2024-05-17T04:35:03.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff601d9b700 1 -- 172.21.15.50:0/1230957608 learned_addr learned my addr 172.21.15.50:0/1230957608 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff602d9d700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 0x7ff60419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.061+0000 7ff60259c700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 0x7ff60419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 -- 172.21.15.50:0/1230957608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 msgr2=0x7ff60419b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 0x7ff60419b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 -- 172.21.15.50:0/1230957608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 msgr2=0x7ff60419bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 0x7ff60419bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 -- 172.21.15.50:0/1230957608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6041a6810 con 0x7ff604107730 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff60259c700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 0x7ff60419b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff601d9b700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c8f0 tx=0x7ff5f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff5f37fe700 1 -- 172.21.15.50:0/1230957608 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8007940 con 0x7ff604107730 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff5f37fe700 1 -- 172.21.15.50:0/1230957608 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f800f450 con 0x7ff604107730 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff5f37fe700 1 -- 172.21.15.50:0/1230957608 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8018660 con 0x7ff604107730 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6041a6b00 con 0x7ff604107730 2024-05-17T04:35:03.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.062+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041a7040 con 0x7ff604107730 2024-05-17T04:35:03.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.065+0000 7ff5f37fe700 1 -- 172.21.15.50:0/1230957608 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff5f8018800 con 0x7ff604107730 2024-05-17T04:35:03.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff5f37fe700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 0x7ff5ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff617354b80 1 Processor -- start 2024-05-17T04:35:03.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff60259c700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 0x7ff5ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff5f37fe700 1 -- 172.21.15.50:0/1230957608 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff5f80e2480 con 0x7ff604107730 2024-05-17T04:35:03.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff617354b80 1 -- start start 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff617354b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff617354b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a3cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff617354b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3a3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.066+0000 7ff60259c700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 0x7ff5ec0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009750 tx=0x7ff5f400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff617354b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afd396b240 con 0x55afd38be9a0 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff617354b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55afd395af80 con 0x55afd3a3d120 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff617354b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afd395ee20 con 0x55afd3a42a50 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55736/0 (socket says 172.21.15.50:55736) 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 -- 172.21.15.50:0/1057253399 learned_addr learned my addr 172.21.15.50:0/1057253399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff601d9b700 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a3cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff602d9d700 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3a3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 -- 172.21.15.50:0/1057253399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 msgr2=0x55afd3a3cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a3cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 -- 172.21.15.50:0/1057253399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 msgr2=0x55afd3a3d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3a3d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 -- 172.21.15.50:0/1057253399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd3904540 con 0x55afd38be9a0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff60259c700 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40055e0 tx=0x7ff5f401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7f7d39efefa118 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff5f0ff9700 1 -- 172.21.15.50:0/1057253399 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4005610 con 0x55afd38be9a0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff5f0ff9700 1 -- 172.21.15.50:0/1057253399 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f400bc60 con 0x55afd38be9a0 2024-05-17T04:35:03.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.067+0000 7ff5f0ff9700 1 -- 172.21.15.50:0/1057253399 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400fb20 con 0x55afd38be9a0 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 -- 172.21.15.50:0/1057253399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 msgr2=0x55afd3a42510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40055e0 tx=0x7ff5f401d000 comp rx=0 tx=0).stop 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 -- 172.21.15.50:0/1057253399 shutdown_connections 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3a3d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a42510 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 --2- 172.21.15.50:0/1057253399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a3cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.068+0000 7ff617354b80 1 -- 172.21.15.50:0/1057253399 >> 172.21.15.50:0/1057253399 conn(0x55afd3902eb0 msgr2=0x55afd3a37b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- 172.21.15.50:0/1057253399 shutdown_connections 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- 172.21.15.50:0/1057253399 wait complete. 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- start start 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 Processor -- start 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 0x55afd3a46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afd3862100 con 0x55afd3a3d120 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55afd38fbe30 con 0x55afd38be9a0 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a49340 con 0x55afd3a42a50 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff617354b80 10 client.0 _ll_register_callbacks cb 0x55afd38f5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff601d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff601d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55752/0 (socket says 172.21.15.50:55752) 2024-05-17T04:35:03.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.069+0000 7ff601d9b700 1 -- 172.21.15.50:0/2942739432 learned_addr learned my addr 172.21.15.50:0/2942739432 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff602d9d700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff60259c700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 0x55afd3a46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 -- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 msgr2=0x55afd3a46c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a46c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 -- 172.21.15.50:0/2942739432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 msgr2=0x55afd3a46180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 0x55afd3a46180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 -- 172.21.15.50:0/2942739432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd38ab430 con 0x55afd3a3d120 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff60259c700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 0x55afd3a46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff601d9b700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80a9410 tx=0x7ff5f80cc140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8645021421b024 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80a9710 con 0x55afd3a3d120 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f400bc60 con 0x55afd3a3d120 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff5f80a5d00 con 0x55afd3a3d120 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5f806f170 con 0x55afd3a3d120 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff617354b80 10 client.15093 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:03.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.070+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afd3a53dc0 con 0x55afd3a3d120 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f806a020 con 0x55afd3a3d120 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f80a9970 con 0x55afd3a3d120 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 20 client.15093 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3a57820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 4 client.15093 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 --> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd396d190 con 0x55afd3a563b0 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.071+0000 7ff60259c700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3a57820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.072+0000 7ff60259c700 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3a57820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2956c98b5c395287 server_cookie=9cfeec8b54be2f68 in_seq=0 out_seq=0 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.072+0000 7ff5e9ffb700 10 client.15093 ms_handle_connect on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/2942739432 <== mds.1 v2:172.21.15.110:6838/2082916819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afd396d190 con 0x55afd3a563b0 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff5e9ffb700 10 client.15093 handle_command_reply: tid=0 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 client.15093 shutdown 2024-05-17T04:35:03.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 msgr2=0x55afd3a466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80a9410 tx=0x7ff5f80cc140 comp rx=0 tx=0).stop 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 shutdown_connections 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3a57820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff5e9ffb700 0 client.15093 ms_handle_reset on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a46c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd38be9a0 0x55afd3a46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 --2- 172.21.15.50:0/2942739432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd3a3d120 0x55afd3a466c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 >> 172.21.15.50:0/2942739432 conn(0x55afd3902eb0 msgr2=0x55afd3a31e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 shutdown_connections 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 1 -- 172.21.15.50:0/2942739432 wait complete. 2024-05-17T04:35:03.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.073+0000 7ff617354b80 20 client.15093 trim_cache size 0 max 16384 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 Processor -- start 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 -- start start 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a3d120 0x55afd3a31e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a42a50 0x55afd3a403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a55b60 con 0x55afd38be9a0 2024-05-17T04:35:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a56230 con 0x55afd3a42a50 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.095+0000 7ff617354b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a56900 con 0x55afd3a3d120 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55760/0 (socket says 172.21.15.50:55760) 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 -- 172.21.15.50:0/303178368 learned_addr learned my addr 172.21.15.50:0/303178368 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 -- 172.21.15.50:0/303178368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a3d120 msgr2=0x55afd3a31e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff601d9b700 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a3d120 0x55afd3a31e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a3d120 0x55afd3a31e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 -- 172.21.15.50:0/303178368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a42a50 msgr2=0x55afd3a403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a42a50 0x55afd3a403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 -- 172.21.15.50:0/303178368 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd3a42080 con 0x55afd38be9a0 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff60259c700 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4003e30 tx=0x7ff5f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c74d93b44f7eb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff5ebfff700 1 -- 172.21.15.50:0/303178368 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4033040 con 0x55afd38be9a0 2024-05-17T04:35:03.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff5ebfff700 1 -- 172.21.15.50:0/303178368 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f403f070 con 0x55afd38be9a0 2024-05-17T04:35:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.096+0000 7ff5ebfff700 1 -- 172.21.15.50:0/303178368 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4039730 con 0x55afd38be9a0 2024-05-17T04:35:03.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- 172.21.15.50:0/303178368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 msgr2=0x55afd3a3cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4003e30 tx=0x7ff5f4000bc0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- 172.21.15.50:0/303178368 shutdown_connections 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a42a50 0x55afd3a403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3a3cbe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 --2- 172.21.15.50:0/303178368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a3d120 0x55afd3a31e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- 172.21.15.50:0/303178368 >> 172.21.15.50:0/303178368 conn(0x55afd3902eb0 msgr2=0x55afd3a3fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- 172.21.15.50:0/303178368 shutdown_connections 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- 172.21.15.50:0/303178368 wait complete. 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.097+0000 7ff617354b80 1 -- start start 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 Processor -- start 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3ab9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a576a0 con 0x55afd38be9a0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a56fd0 con 0x55afd3a3d120 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afd3a57d70 con 0x55afd3a42a50 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff617354b80 10 client.0 _ll_register_callbacks cb 0x55afd38f5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55774/0 (socket says 172.21.15.50:55774) 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 -- 172.21.15.50:0/361611305 learned_addr learned my addr 172.21.15.50:0/361611305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff601d9b700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3ab9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff602d9d700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 -- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 msgr2=0x55afd3a60740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a60740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 -- 172.21.15.50:0/361611305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 msgr2=0x55afd3ab9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3ab9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff60259c700 1 -- 172.21.15.50:0/361611305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd3a60d80 con 0x55afd38be9a0 2024-05-17T04:35:03.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.098+0000 7ff602d9d700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff60259c700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009990 tx=0x7ff5f400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf94b91cfbba2b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4047070 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff5f400f860 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f403f070 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5f4053050 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 10 client.15099 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afd3ac11e0 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5f403f070 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f402c070 con 0x55afd38be9a0 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.099+0000 7ff617354b80 20 client.15099 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:03.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3ac4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff617354b80 4 client.15099 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 --> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] -- command(tid 0: {"prefix": "status"}) -- 0x55afd3966970 con 0x55afd3a563b0 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff601d9b700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3ac4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff601d9b700 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3ac4590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f588a057e3969f4 server_cookie=4be01317988bdfb8 in_seq=0 out_seq=0 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.100+0000 7ff5e9ffb700 10 client.15099 ms_handle_connect on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff5e9ffb700 1 -- 172.21.15.50:0/361611305 <== mds.1 v2:172.21.15.110:6838/2082916819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55afd3966970 con 0x55afd3a563b0 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff5e9ffb700 10 client.15099 handle_command_reply: tid=0 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 client.15099 shutdown 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 msgr2=0x55afd3ab99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009990 tx=0x7ff5f400b9c0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 shutdown_connections 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x55afd3a563b0 0x55afd3ac4590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff5e9ffb700 0 client.15099 ms_handle_reset on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:03.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afd3a42a50 0x55afd3a60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55afd38be9a0 0x55afd3ab99e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 --2- 172.21.15.50:0/361611305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55afd3a3d120 0x55afd3ab9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 >> 172.21.15.50:0/361611305 conn(0x55afd3902eb0 msgr2=0x55afd3a43a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 shutdown_connections 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 1 -- 172.21.15.50:0/361611305 wait complete. 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.101+0000 7ff617354b80 20 client.15099 trim_cache size 0 max 16384 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24499, 2024-05-17T04:35:03.106 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 136.07502199300001, 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 224.595776038 2024-05-17T04:35:03.107 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 msgr2=0x7ff5ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 0x7ff5ec0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009750 tx=0x7ff5f400b580 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 msgr2=0x7ff60419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c8f0 tx=0x7ff5f800cc00 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 shutdown_connections 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff604108080 0x7ff60419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff604102e00 0x7ff60419b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7ff5ec0b2880 0x7ff5ec0b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 --2- 172.21.15.50:0/1230957608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff604107730 0x7ff60419ba90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 >> 172.21.15.50:0/1230957608 conn(0x7ff604079bd0 msgr2=0x7ff604106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 shutdown_connections 2024-05-17T04:35:03.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:03.102+0000 7ff608979700 1 -- 172.21.15.50:0/1230957608 wait complete. 2024-05-17T04:35:03.841 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T04:35:04.098 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:04.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:04 smithi105 ceph-mon[34795]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:04.374 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:04 smithi050 ceph-mon[31739]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:04 smithi110 ceph-mon[35497]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:05.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/1587555173 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8108840 msgr2=0x7fe4d8108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/1587555173 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8108840 0x7fe4d8108c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c0009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/1587555173 shutdown_connections 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/1587555173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8109190 0x7fe4d8110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/1587555173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8103f10 0x7fe4d8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/1587555173 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8108840 0x7fe4d8108c50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/1587555173 >> 172.21.15.50:0/1587555173 conn(0x7fe4d80ff9b0 msgr2=0x7fe4d8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/1587555173 shutdown_connections 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.865+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/1587555173 wait complete. 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 Processor -- start 2024-05-17T04:35:05.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 -- start start 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8108840 0x7fe4d819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8109190 0x7fe4d819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8113c50 con 0x7fe4d8103f10 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8113ad0 con 0x7fe4d8109190 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4dd4cf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8113dd0 con 0x7fe4d8108840 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55800/0 (socket says 172.21.15.50:55800) 2024-05-17T04:35:05.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/2569697876 learned_addr learned my addr 172.21.15.50:0/2569697876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4cffff700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8108840 0x7fe4d819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8109190 0x7fe4d819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/2569697876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8108840 msgr2=0x7fe4d819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8108840 0x7fe4d819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/2569697876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8109190 msgr2=0x7fe4d819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8109190 0x7fe4d819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.866+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/2569697876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d819f600 con 0x7fe4d8103f10 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d950 tx=0x7fe4c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4d4ff9700 1 -- 172.21.15.50:0/2569697876 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8016070 con 0x7fe4d8103f10 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4d4ff9700 1 -- 172.21.15.50:0/2569697876 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c800fd20 con 0x7fe4d8103f10 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d819f8f0 con 0x7fe4d8103f10 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4d4ff9700 1 -- 172.21.15.50:0/2569697876 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800e3b0 con 0x7fe4d8103f10 2024-05-17T04:35:05.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.867+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81a7210 con 0x7fe4d8103f10 2024-05-17T04:35:05.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.871+0000 7fe4d4ff9700 1 -- 172.21.15.50:0/2569697876 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe4c8019070 con 0x7fe4d8103f10 2024-05-17T04:35:05.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.871+0000 7fe4d4ff9700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 0x7fe4c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.871+0000 7fe4cffff700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 0x7fe4c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.871+0000 7fe4d4ff9700 1 -- 172.21.15.50:0/2569697876 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4c80e1fb0 con 0x7fe4d8103f10 2024-05-17T04:35:05.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 Processor -- start 2024-05-17T04:35:05.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 -- start start 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4cffff700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 0x7fe4c40b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006010 tx=0x7fe4c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a8b2ec080 con 0x558a8b3c76a0 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b2dbc20 con 0x558a8b3c1ec0 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558a8b2dfc60 con 0x558a8b23f9a0 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.872+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55802/0 (socket says 172.21.15.50:55802) 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 -- 172.21.15.50:0/1610943704 learned_addr learned my addr 172.21.15.50:0/1610943704 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 -- 172.21.15.50:0/1610943704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 msgr2=0x558a8b3c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 -- 172.21.15.50:0/1610943704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 msgr2=0x558a8b3c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 -- 172.21.15.50:0/1610943704 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8b275570 con 0x558a8b3c76a0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38992e52bf360ba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cd7fa700 1 -- 172.21.15.50:0/1610943704 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000bbb0 con 0x558a8b3c76a0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cd7fa700 1 -- 172.21.15.50:0/1610943704 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c000bd50 con 0x558a8b3c76a0 2024-05-17T04:35:05.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.873+0000 7fe4cd7fa700 1 -- 172.21.15.50:0/1610943704 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c000f9e0 con 0x558a8b3c76a0 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1610943704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 msgr2=0x558a8b3c1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c001e000 comp rx=0 tx=0).stop 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1610943704 shutdown_connections 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1610943704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3c1980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1610943704 >> 172.21.15.50:0/1610943704 conn(0x558a8b29f780 msgr2=0x558a8b3b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.874+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1610943704 shutdown_connections 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1610943704 wait complete. 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 -- start start 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 Processor -- start 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 0x558a8b3caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a8b1e3100 con 0x558a8b3c1ec0 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b27d8f0 con 0x558a8b23f9a0 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3ce010 con 0x558a8b3c76a0 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.875+0000 7fe4ebeaab80 10 client.0 _ll_register_callbacks cb 0x558a8b272b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55808/0 (socket says 172.21.15.50:55808) 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 -- 172.21.15.50:0/1750411127 learned_addr learned my addr 172.21.15.50:0/1750411127 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 0x558a8b3caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 -- 172.21.15.50:0/1750411127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 msgr2=0x558a8b3d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 -- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 msgr2=0x558a8b3caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 0x558a8b3caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 -- 172.21.15.50:0/1750411127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8b22c430 con 0x558a8b3c1ec0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 0x558a8b3caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4cffff700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098e0 tx=0x7fe4c000f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b145105195754a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:05.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c003c060 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4c00039d0 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.876+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c000bd50 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4c000bbb0 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 10 client.15111 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a8b3d8b30 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c000bd50 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe4c0022070 con 0x558a8b3c1ec0 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 10 client.15111 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 20 client.15111 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b3d9e80 0x558a8b3dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 4 client.15111 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:05.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a8b2edfd0 con 0x558a8b3d9e80 2024-05-17T04:35:05.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.877+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b3d9e80 0x558a8b3dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.878+0000 7fe4be7fc700 10 client.15111 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:05.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.878+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b3d9e80 0x558a8b3dbfd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90b2992138274ca server_cookie=31e78429658e26fc in_seq=0 out_seq=0 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4be7fc700 1 -- 172.21.15.50:0/1750411127 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a8b2edfd0 con 0x558a8b3d9e80 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4be7fc700 10 client.15111 handle_command_reply: tid=0 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 client.15111 shutdown 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 msgr2=0x558a8b3cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098e0 tx=0x7fe4c000f960 comp rx=0 tx=0).stop 2024-05-17T04:35:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 shutdown_connections 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b3d9e80 0x558a8b3dbfd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b23f9a0 0x558a8b3caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4be7fc700 0 client.15111 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/1750411127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 >> 172.21.15.50:0/1750411127 conn(0x558a8b29f780 msgr2=0x558a8b3b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 shutdown_connections 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/1750411127 wait complete. 2024-05-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.879+0000 7fe4ebeaab80 20 client.15111 trim_cache size 0 max 16384 2024-05-17T04:35:05.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.900+0000 7fe4ebeaab80 1 Processor -- start 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.900+0000 7fe4ebeaab80 1 -- start start 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3da310 con 0x558a8b23f9a0 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3da9e0 con 0x558a8b3c1ec0 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3db0b0 con 0x558a8b3c76a0 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55820/0 (socket says 172.21.15.50:55820) 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/695334298 learned_addr learned my addr 172.21.15.50:0/695334298 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:05.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4cffff700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/695334298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 msgr2=0x558a8b3c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/695334298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 msgr2=0x558a8b3cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 -- 172.21.15.50:0/695334298 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8b3b4bd0 con 0x558a8b23f9a0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4cffff700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.901+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a91d0 tx=0x7fe4c8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77983e917a41bec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.902+0000 7fe4ccff9700 1 -- 172.21.15.50:0/695334298 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8020040 con 0x558a8b23f9a0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.902+0000 7fe4ccff9700 1 -- 172.21.15.50:0/695334298 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c80a5830 con 0x558a8b23f9a0 2024-05-17T04:35:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.902+0000 7fe4ccff9700 1 -- 172.21.15.50:0/695334298 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a9700 con 0x558a8b23f9a0 2024-05-17T04:35:05.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.902+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/695334298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 msgr2=0x558a8b3b8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.902+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a91d0 tx=0x7fe4c8005dc0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/695334298 shutdown_connections 2024-05-17T04:35:05.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b3c76a0 0x558a8b3c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b23f9a0 0x558a8b3b8240 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/695334298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/695334298 >> 172.21.15.50:0/695334298 conn(0x558a8b29f780 msgr2=0x558a8b3c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/695334298 shutdown_connections 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/695334298 wait complete. 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- start start 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 Processor -- start 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3dbe50 con 0x558a8b3c76a0 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3db780 con 0x558a8b3c1ec0 2024-05-17T04:35:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558a8b3dc520 con 0x558a8b23f9a0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.903+0000 7fe4ebeaab80 10 client.0 _ll_register_callbacks cb 0x558a8b272b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55834/0 (socket says 172.21.15.50:55834) 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 -- 172.21.15.50:0/3115307424 learned_addr learned my addr 172.21.15.50:0/3115307424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4cffff700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 -- 172.21.15.50:0/3115307424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 msgr2=0x558a8b3d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 -- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 msgr2=0x558a8b3d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 -- 172.21.15.50:0/3115307424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8b3e3bb0 con 0x558a8b3c76a0 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4d77fe700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0016330 tx=0x7fe4d0016360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8903689a59c74b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0016e20 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4d0018410 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.904+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c80a5830 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0029050 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 10 client.15117 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a8b446320 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c80a5830 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe4d00176a0 con 0x558a8b3c76a0 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 20 client.15117 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b448800 0x558a8b449c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 4 client.15117 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "status"}) -- 0x558a8b2e77b0 con 0x558a8b448800 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.905+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b448800 0x558a8b449c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4be7fc700 10 client.15117 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4d6ffd700 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b448800 0x558a8b449c70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909de6ca9ac1bc9 server_cookie=6a2255863fb1ffd8 in_seq=0 out_seq=0 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4be7fc700 1 -- 172.21.15.50:0/3115307424 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558a8b2e77b0 con 0x558a8b448800 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4be7fc700 10 client.15117 handle_command_reply: tid=0 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 client.15117 shutdown 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 msgr2=0x558a8b3d7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0016330 tx=0x7fe4d0016360 comp rx=0 tx=0).stop 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 shutdown_connections 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a8b3c76a0 0x558a8b3d7540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558a8b448800 0x558a8b449c70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4be7fc700 0 client.15117 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558a8b23f9a0 0x558a8b3d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 --2- 172.21.15.50:0/3115307424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b3c1ec0 0x558a8b3d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 >> 172.21.15.50:0/3115307424 conn(0x558a8b29f780 msgr2=0x558a8b3c4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 shutdown_connections 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.906+0000 7fe4ebeaab80 1 -- 172.21.15.50:0/3115307424 wait complete. 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4ebeaab80 20 client.15117 trim_cache size 0 max 16384 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-17T04:35:05.912 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24506, 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 199.98244832699999, 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 217.235991673 2024-05-17T04:35:05.913 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 msgr2=0x7fe4c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 0x7fe4c40b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0006010 tx=0x7fe4c0005960 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 msgr2=0x7fe4d819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d950 tx=0x7fe4c800dc60 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 shutdown_connections 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d8109190 0x7fe4d819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4c40b2900 0x7fe4c40b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4d8103f10 0x7fe4d819a730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 --2- 172.21.15.50:0/2569697876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d8108840 0x7fe4d819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:05.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 >> 172.21.15.50:0/2569697876 conn(0x7fe4d80ff9b0 msgr2=0x7fe4d8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:05.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 shutdown_connections 2024-05-17T04:35:05.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:05.907+0000 7fe4dd4cf700 1 -- 172.21.15.50:0/2569697876 wait complete. 2024-05-17T04:35:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:06 smithi105 ceph-mon[34795]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:06.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:06 smithi110 ceph-mon[35497]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:06 smithi050 ceph-mon[31739]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:06.728 INFO:tasks.cephadm:Setup complete, yielding 2024-05-17T04:35:06.729 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T04:35:06.738 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:35:06.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-17T04:35:06.996 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:08.407 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:08 smithi050 ceph-mon[31739]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:08 smithi105 ceph-mon[34795]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:08 smithi110 ceph-mon[35497]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4221247087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 msgr2=0x7fa7bc104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4221247087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc104150 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80099c0 tx=0x7fa7b8009cd0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4221247087 shutdown_connections 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4221247087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 0x7fa7bc10c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4221247087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc104150 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4221247087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4221247087 >> 172.21.15.50:0/4221247087 conn(0x7fa7bc0ff550 msgr2=0x7fa7bc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4221247087 shutdown_connections 2024-05-17T04:35:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.865+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4221247087 wait complete. 2024-05-17T04:35:08.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 Processor -- start 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 -- start start 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 0x7fa7bc0760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10f640 con 0x7fa7bc103d40 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10f4c0 con 0x7fa7bc104e70 2024-05-17T04:35:08.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.866+0000 7fa7c3fd8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc10f7c0 con 0x7fa7bc105810 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52588/0 (socket says 172.21.15.50:52588) 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 -- 172.21.15.50:0/4259852038 learned_addr learned my addr 172.21.15.50:0/4259852038 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1573700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c2575700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 0x7fa7bc0760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 -- 172.21.15.50:0/4259852038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 msgr2=0x7fa7bc0760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 0x7fa7bc0760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 -- 172.21.15.50:0/4259852038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 msgr2=0x7fa7bc075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 -- 172.21.15.50:0/4259852038 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc076820 con 0x7fa7bc103d40 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1573700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c1d74700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800fa80 tx=0x7fa7b800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8027070 con 0x7fa7bc103d40 2024-05-17T04:35:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7b8017a90 con 0x7fa7bc103d40 2024-05-17T04:35:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc1a42f0 con 0x7fa7bc103d40 2024-05-17T04:35:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8009e60 con 0x7fa7bc103d40 2024-05-17T04:35:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.867+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1a46f0 con 0x7fa7bc103d40 2024-05-17T04:35:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.868+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7bc104150 con 0x7fa7bc103d40 2024-05-17T04:35:08.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.871+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa7b802a070 con 0x7fa7bc103d40 2024-05-17T04:35:08.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.872+0000 7fa7b2ffd700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 0x7fa7a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.872+0000 7fa7c1573700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 0x7fa7a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.872+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa7b80e31b0 con 0x7fa7bc103d40 2024-05-17T04:35:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.872+0000 7fa7c1573700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 0x7fa7a80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac005950 tx=0x7fa7ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:08.875+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7b8013070 con 0x7fa7bc103d40 2024-05-17T04:35:09.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.108+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa7bc064700 con 0x7fa7a80b2880 2024-05-17T04:35:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.109+0000 7fa7b2ffd700 1 -- 172.21.15.50:0/4259852038 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa7bc064700 con 0x7fa7a80b2880 2024-05-17T04:35:09.109 INFO:teuthology.orchestra.run.smithi050.stdout:Backend: cephadm 2024-05-17T04:35:09.109 INFO:teuthology.orchestra.run.smithi050.stdout:Available: Yes 2024-05-17T04:35:09.109 INFO:teuthology.orchestra.run.smithi050.stdout:Paused: No 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 msgr2=0x7fa7a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 0x7fa7a80b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac005950 tx=0x7fa7ac0058e0 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 msgr2=0x7fa7bc0774d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800fa80 tx=0x7fa7b800fb60 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 shutdown_connections 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.111+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa7bc105810 0x7fa7bc0760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa7a80b2880 0x7fa7a80b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa7bc103d40 0x7fa7bc0774d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 --2- 172.21.15.50:0/4259852038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc104e70 0x7fa7bc075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 >> 172.21.15.50:0/4259852038 conn(0x7fa7bc0ff550 msgr2=0x7fa7bc108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 shutdown_connections 2024-05-17T04:35:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:09.112+0000 7fa7c3fd8700 1 -- 172.21.15.50:0/4259852038 wait complete. 2024-05-17T04:35:09.872 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-17T04:35:10.133 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:10.445 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:10 smithi050 ceph-mon[31739]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:10.445 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:10 smithi050 ceph-mon[31739]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:10 smithi105 ceph-mon[34795]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:10.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:10 smithi105 ceph-mon[34795]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:10 smithi110 ceph-mon[35497]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:10 smithi110 ceph-mon[35497]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:11.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 -- 172.21.15.50:0/1254078153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000103820 msgr2=0x7f6000103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 --2- 172.21.15.50:0/1254078153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000103820 0x7f6000103c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800b0a0 tx=0x7f5ff800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 -- 172.21.15.50:0/1254078153 shutdown_connections 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 --2- 172.21.15.50:0/1254078153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000103820 0x7f6000103c30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 --2- 172.21.15.50:0/1254078153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000102580 0x7f6000102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 --2- 172.21.15.50:0/1254078153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102ed0 0x7f60001032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 -- 172.21.15.50:0/1254078153 >> 172.21.15.50:0/1254078153 conn(0x7f60000fe3c0 msgr2=0x7f6000100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 -- 172.21.15.50:0/1254078153 shutdown_connections 2024-05-17T04:35:11.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.978+0000 7f60081a4700 1 -- 172.21.15.50:0/1254078153 wait complete. 2024-05-17T04:35:11.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 Processor -- start 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 -- start start 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102580 0x7f600019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000103820 0x7f60001a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60001117b0 con 0x7f6000102ed0 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6000111630 con 0x7f6000103820 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f60081a4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6000111930 con 0x7f6000102580 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.979+0000 7f600573f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52610/0 (socket says 172.21.15.50:52610) 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 -- 172.21.15.50:0/738963162 learned_addr learned my addr 172.21.15.50:0/738963162 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:11.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f6005f40700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102580 0x7f600019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f6006741700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000103820 0x7f60001a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 -- 172.21.15.50:0/738963162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102580 msgr2=0x7f600019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102580 0x7f600019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 -- 172.21.15.50:0/738963162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000103820 msgr2=0x7f60001a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000103820 0x7f60001a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 -- 172.21.15.50:0/738963162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60001a3ea0 con 0x7f6000102ed0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f600573f700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000d950 tx=0x7f5ff000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0016070 con 0x7f6000102ed0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ff000fd20 con 0x7f6000102ed0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60001a4190 con 0x7f6000102ed0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000e3b0 con 0x7f6000102ed0 2024-05-17T04:35:11.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.980+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60001a4720 con 0x7f6000102ed0 2024-05-17T04:35:11.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.981+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60001032e0 con 0x7f6000102ed0 2024-05-17T04:35:11.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.983+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ff0019070 con 0x7f6000102ed0 2024-05-17T04:35:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.984+0000 7f5ff6ffd700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 0x7f5fec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.984+0000 7f6005f40700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 0x7f5fec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.984+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ff00e1fe0 con 0x7f6000102ed0 2024-05-17T04:35:11.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.984+0000 7f6005f40700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 0x7f5fec0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f600006c0a0 tx=0x7f5ffc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:11.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:11.987+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ff00a9950 con 0x7f6000102ed0 2024-05-17T04:35:12.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.211+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6000064770 con 0x7f5fec0b2900 2024-05-17T04:35:12.211 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:12 smithi050 ceph-mon[31739]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:12.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.225+0000 7f5ff6ffd700 1 -- 172.21.15.50:0/738963162 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6000064770 con 0x7f5fec0b2900 2024-05-17T04:35:12.225 INFO:teuthology.orchestra.run.smithi050.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-17T04:35:12.225 INFO:teuthology.orchestra.run.smithi050.stdout:mds.a smithi050 running (3m) 49s ago 3m 82.0M - 19.0.0-3794-g931809a9 8880f7613d02 05accb6677ec 2024-05-17T04:35:12.225 INFO:teuthology.orchestra.run.smithi050.stdout:mds.b smithi105 running (3m) 87s ago 3m 92.9M - 19.0.0-3794-g931809a9 8880f7613d02 68bf080cc3aa 2024-05-17T04:35:12.225 INFO:teuthology.orchestra.run.smithi050.stdout:mds.c smithi110 running (4m) 87s ago 3m 85.3M - 19.0.0-3794-g931809a9 8880f7613d02 92bad0877e04 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.d smithi050 running (4m) 49s ago 4m 84.7M - 19.0.0-3794-g931809a9 8880f7613d02 d5133a926436 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.e smithi105 running (4m) 87s ago 4m 89.5M - 19.0.0-3794-g931809a9 8880f7613d02 f84decd82357 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.f smithi110 running (4m) 87s ago 4m 84.9M - 19.0.0-3794-g931809a9 8880f7613d02 f8b014e3ee74 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.g smithi050 running (3m) 49s ago 3m 81.1M - 19.0.0-3794-g931809a9 8880f7613d02 bd18c3f0e4ae 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.h smithi105 running (3m) 87s ago 3m 85.5M - 19.0.0-3794-g931809a9 8880f7613d02 fdb48be1149d 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.i smithi110 running (3m) 87s ago 3m 85.3M - 19.0.0-3794-g931809a9 8880f7613d02 8a4dee2609fc 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.j smithi050 running (4m) 49s ago 4m 84.5M - 19.0.0-3794-g931809a9 8880f7613d02 2b852a49f758 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.k smithi105 running (4m) 87s ago 4m 82.2M - 19.0.0-3794-g931809a9 8880f7613d02 fed180584642 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mds.l smithi110 running (4m) 87s ago 4m 84.7M - 19.0.0-3794-g931809a9 8880f7613d02 fe04b9cc6b4d 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.x smithi050 *:9283,8765 running (11m) 49s ago 11m 599M - 19.0.0-3794-g931809a9 8880f7613d02 5b7105165780 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.y smithi105 *:8443,8765 running (9m) 87s ago 9m 537M - 19.0.0-3794-g931809a9 8880f7613d02 2b4c477648d0 2024-05-17T04:35:12.226 INFO:teuthology.orchestra.run.smithi050.stdout:mgr.z smithi110 *:8443,8765 running (9m) 87s ago 9m 536M - 19.0.0-3794-g931809a9 8880f7613d02 169a9b32a072 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:mon.a smithi050 running (11m) 49s ago 11m 119M 2048M 19.0.0-3794-g931809a9 8880f7613d02 ae0dc175e3a3 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:mon.b smithi105 running (10m) 87s ago 10m 110M 2048M 19.0.0-3794-g931809a9 8880f7613d02 f78e990643ec 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:mon.c smithi110 running (10m) 87s ago 10m 111M 2048M 19.0.0-3794-g931809a9 8880f7613d02 0271de19e04b 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 smithi050 running (9m) 49s ago 9m 124M 4273M 19.0.0-3794-g931809a9 8880f7613d02 1bb80c3b0d97 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 smithi105 running (9m) 87s ago 9m 113M 4272M 19.0.0-3794-g931809a9 8880f7613d02 6c0baee0344a 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 smithi110 running (8m) 87s ago 8m 117M 4272M 19.0.0-3794-g931809a9 8880f7613d02 d0dd6f2dee21 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 smithi050 running (8m) 49s ago 8m 115M 4273M 19.0.0-3794-g931809a9 8880f7613d02 c3582222ede8 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 smithi105 running (7m) 87s ago 7m 110M 4272M 19.0.0-3794-g931809a9 8880f7613d02 c321e5b5aa11 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 smithi110 running (7m) 87s ago 7m 125M 4272M 19.0.0-3794-g931809a9 8880f7613d02 05c41517a1d6 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 smithi050 running (6m) 49s ago 6m 112M 4273M 19.0.0-3794-g931809a9 8880f7613d02 9b18207624dd 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 smithi105 running (6m) 87s ago 6m 111M 4272M 19.0.0-3794-g931809a9 8880f7613d02 62bc3bdce0fc 2024-05-17T04:35:12.227 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 smithi110 running (6m) 87s ago 6m 123M 4272M 19.0.0-3794-g931809a9 8880f7613d02 9a3ecf4a23a1 2024-05-17T04:35:12.228 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 smithi050 running (5m) 49s ago 5m 121M 4273M 19.0.0-3794-g931809a9 8880f7613d02 3e1bea69b00a 2024-05-17T04:35:12.228 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 smithi105 running (5m) 87s ago 5m 111M 4272M 19.0.0-3794-g931809a9 8880f7613d02 adee554ee033 2024-05-17T04:35:12.228 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 smithi110 running (4m) 87s ago 4m 123M 4272M 19.0.0-3794-g931809a9 8880f7613d02 e66c77b80e78 2024-05-17T04:35:12.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.227+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 msgr2=0x7f5fec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.227+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 0x7f5fec0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f600006c0a0 tx=0x7f5ffc00b410 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.227+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 msgr2=0x7f600019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.227+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000d950 tx=0x7f5ff000dc60 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 shutdown_connections 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6000103820 0x7f60001a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6000102580 0x7f600019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5fec0b2900 0x7f5fec0b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 --2- 172.21.15.50:0/738963162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6000102ed0 0x7f600019c880 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 >> 172.21.15.50:0/738963162 conn(0x7f60000fe3c0 msgr2=0x7f6000100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 shutdown_connections 2024-05-17T04:35:12.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:12.228+0000 7f60081a4700 1 -- 172.21.15.50:0/738963162 wait complete. 2024-05-17T04:35:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:12 smithi105 ceph-mon[34795]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:12 smithi110 ceph-mon[35497]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:13.001 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-17T04:35:13.258 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:14.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:14 smithi050 ceph-mon[31739]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:14.458 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:14 smithi050 ceph-mon[31739]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:14 smithi105 ceph-mon[34795]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:14.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:14 smithi105 ceph-mon[34795]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:14 smithi110 ceph-mon[35497]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:14 smithi110 ceph-mon[35497]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 -- 172.21.15.50:0/1579527114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a081099b0 msgr2=0x7f3a0810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 --2- 172.21.15.50:0/1579527114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a081099b0 0x7f3a0810bdd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000b0a0 tx=0x7f3a0000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 -- 172.21.15.50:0/1579527114 shutdown_connections 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 --2- 172.21.15.50:0/1579527114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a081099b0 0x7f3a0810bdd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 --2- 172.21.15.50:0/1579527114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081018a0 0x7f3a08101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 --2- 172.21.15.50:0/1579527114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a08102280 0x7f3a08109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 -- 172.21.15.50:0/1579527114 >> 172.21.15.50:0/1579527114 conn(0x7f3a080fd6e0 msgr2=0x7f3a080ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 -- 172.21.15.50:0/1579527114 shutdown_connections 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.079+0000 7f3a0de27700 1 -- 172.21.15.50:0/1579527114 wait complete. 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 Processor -- start 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 -- start start 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a081018a0 0x7f3a08104cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 0x7f3a08103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 0x7f3a081038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810f590 con 0x7f3a08102280 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810f410 con 0x7f3a081099b0 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a0de27700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810f710 con 0x7f3a081018a0 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f39feffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 0x7f3a08103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:15.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 0x7f3a081038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f39feffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 0x7f3a08103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52628/0 (socket says 172.21.15.50:52628) 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f39feffd700 1 -- 172.21.15.50:0/191798391 learned_addr learned my addr 172.21.15.50:0/191798391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.080+0000 7f3a077fe700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a081018a0 0x7f3a08104cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 -- 172.21.15.50:0/191798391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a081018a0 msgr2=0x7f3a08104cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a081018a0 0x7f3a08104cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 -- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 msgr2=0x7f3a08103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 0x7f3a08103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 -- 172.21.15.50:0/191798391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a08104000 con 0x7f3a081099b0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a07fff700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 0x7f3a081038a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000b070 tx=0x7f3a00004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00021070 con 0x7f3a081099b0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a00007b80 con 0x7f3a081099b0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a081a41b0 con 0x7f3a081099b0 2024-05-17T04:35:15.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00005040 con 0x7f3a081099b0 2024-05-17T04:35:15.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.081+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a081a45a0 con 0x7f3a081099b0 2024-05-17T04:35:15.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.083+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a08101cb0 con 0x7f3a081099b0 2024-05-17T04:35:15.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.084+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3a00024070 con 0x7f3a081099b0 2024-05-17T04:35:15.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.086+0000 7f3a057fa700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 0x7f39f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:15.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.086+0000 7f3a077fe700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 0x7f39f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:15.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.086+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3a0000f020 con 0x7f3a081099b0 2024-05-17T04:35:15.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.086+0000 7f3a077fe700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 0x7f39f40b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a08102ea0 tx=0x7f39f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:15.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.088+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a0002b030 con 0x7f3a081099b0 2024-05-17T04:35:15.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.341+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3a08064700 con 0x7f39f40b2900 2024-05-17T04:35:15.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.347+0000 7f3a057fa700 1 -- 172.21.15.50:0/191798391 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3a08064700 con 0x7f39f40b2900 2024-05-17T04:35:15.347 INFO:teuthology.orchestra.run.smithi050.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-17T04:35:15.347 INFO:teuthology.orchestra.run.smithi050.stdout:mds.all 12/12 90s ago 4m smithi050=a;smithi050=d;smithi050=g;smithi050=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-17T04:35:15.347 INFO:teuthology.orchestra.run.smithi050.stdout:mgr 3/3 90s ago 9m smithi050=x;smithi105=y;smithi110=z;count:3 2024-05-17T04:35:15.347 INFO:teuthology.orchestra.run.smithi050.stdout:mon 3/3 90s ago 10m smithi050:172.21.15.50=a;smithi105:172.21.15.105=b;smithi110:172.21.15.110=c;count:3 2024-05-17T04:35:15.348 INFO:teuthology.orchestra.run.smithi050.stdout:osd 12 90s ago - 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 msgr2=0x7f39f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 0x7f39f40b4e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a08102ea0 tx=0x7f39f8009260 comp rx=0 tx=0).stop 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 msgr2=0x7f3a081038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 0x7f3a081038a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000b070 tx=0x7f3a00004c80 comp rx=0 tx=0).stop 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 shutdown_connections 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a081099b0 0x7f3a081038a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a081018a0 0x7f3a08104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f39f40b2900 0x7f39f40b4e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 --2- 172.21.15.50:0/191798391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3a08102280 0x7f3a08103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:15.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.349+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 >> 172.21.15.50:0/191798391 conn(0x7f3a080fd6e0 msgr2=0x7f3a080fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:15.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.350+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 shutdown_connections 2024-05-17T04:35:15.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:15.350+0000 7f3a0de27700 1 -- 172.21.15.50:0/191798391 wait complete. 2024-05-17T04:35:16.015 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-17T04:35:16.282 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:16 smithi050 ceph-mon[31739]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:16 smithi105 ceph-mon[34795]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:16.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:16 smithi110 ceph-mon[35497]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:17.469 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:17 smithi050 ceph-mon[31739]: from='client.24950 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:17.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:17 smithi105 ceph-mon[34795]: from='client.24950 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:17.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:17 smithi110 ceph-mon[35497]: from='client.24950 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:18.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.078+0000 7fcde93f4700 1 -- 172.21.15.50:0/1134048601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 msgr2=0x7fcde410be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.078+0000 7fcde93f4700 1 --2- 172.21.15.50:0/1134048601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde410be70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 -- 172.21.15.50:0/1134048601 shutdown_connections 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 --2- 172.21.15.50:0/1134048601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde410be70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 --2- 172.21.15.50:0/1134048601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 0x7fcde4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 --2- 172.21.15.50:0/1134048601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 0x7fcde4109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 -- 172.21.15.50:0/1134048601 >> 172.21.15.50:0/1134048601 conn(0x7fcde40fe3c0 msgr2=0x7fcde4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 -- 172.21.15.50:0/1134048601 shutdown_connections 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 -- 172.21.15.50:0/1134048601 wait complete. 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 Processor -- start 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.079+0000 7fcde93f4700 1 -- start start 2024-05-17T04:35:18.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 0x7fcde419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 0x7fcde419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410f5a0 con 0x7fcde4109a50 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410f420 con 0x7fcde4104790 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde93f4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcde410f720 con 0x7fcde41070f0 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52654/0 (socket says 172.21.15.50:52654) 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 -- 172.21.15.50:0/2120379098 learned_addr learned my addr 172.21.15.50:0/2120379098 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde2ffd700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 0x7fcde419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde27fc700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 0x7fcde419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 -- 172.21.15.50:0/2120379098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 msgr2=0x7fcde419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 0x7fcde419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 -- 172.21.15.50:0/2120379098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 msgr2=0x7fcde419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 0x7fcde419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 -- 172.21.15.50:0/2120379098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde41a1c90 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.080+0000 7fcde37fe700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003820 tx=0x7fcddc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.081+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc027070 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.081+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde41a1f20 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.081+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcddc003f60 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.081+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc022c80 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.081+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde41a2310 con 0x7fcde4109a50 2024-05-17T04:35:18.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.082+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde4107500 con 0x7fcde4109a50 2024-05-17T04:35:18.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.083+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcddc02a070 con 0x7fcde4109a50 2024-05-17T04:35:18.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.084+0000 7fcddbfff700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 0x7fcdd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:18.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.085+0000 7fcde2ffd700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 0x7fcdd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:18.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.085+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcddc0e3d70 con 0x7fcde4109a50 2024-05-17T04:35:18.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.085+0000 7fcde2ffd700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 0x7fcdd00b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009550 tx=0x7fcdd40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:18.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.088+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcddc031080 con 0x7fcde4109a50 2024-05-17T04:35:18.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.311+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fcde4064770 con 0x7fcdd00b2880 2024-05-17T04:35:18.312 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:18 smithi050 ceph-mon[31739]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.312+0000 7fcddbfff700 1 -- 172.21.15.50:0/2120379098 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fcde4064770 con 0x7fcdd00b2880 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stdout:HOST ADDR LABELS STATUS 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 172.21.15.50 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stdout:smithi105 172.21.15.105 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stdout:smithi110 172.21.15.110 2024-05-17T04:35:18.313 INFO:teuthology.orchestra.run.smithi050.stdout:3 hosts in cluster 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 msgr2=0x7fcdd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 0x7fcdd00b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009550 tx=0x7fcdd40091b0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 msgr2=0x7fcde41a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003820 tx=0x7fcddc009580 comp rx=0 tx=0).stop 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 shutdown_connections 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcdd00b2880 0x7fcdd00b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4109a50 0x7fcde41a1610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde4104790 0x7fcde419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 --2- 172.21.15.50:0/2120379098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcde41070f0 0x7fcde419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 >> 172.21.15.50:0/2120379098 conn(0x7fcde40fe3c0 msgr2=0x7fcde4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 shutdown_connections 2024-05-17T04:35:18.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:18.315+0000 7fcde93f4700 1 -- 172.21.15.50:0/2120379098 wait complete. 2024-05-17T04:35:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:18 smithi105 ceph-mon[34795]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:18 smithi110 ceph-mon[35497]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:19.028 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-17T04:35:19.285 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:20.447 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:20 smithi050 ceph-mon[31739]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:20.448 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:20 smithi050 ceph-mon[31739]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:20 smithi105 ceph-mon[34795]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:20 smithi105 ceph-mon[34795]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:20.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:20 smithi110 ceph-mon[35497]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:20.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:20 smithi110 ceph-mon[35497]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.075+0000 7fd009afc700 1 -- 172.21.15.50:0/1929303838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004103d30 msgr2=0x7fd004104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.075+0000 7fd009afc700 1 --2- 172.21.15.50:0/1929303838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004103d30 0x7fd004104140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcff40099c0 tx=0x7fcff4009cd0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 -- 172.21.15.50:0/1929303838 shutdown_connections 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 --2- 172.21.15.50:0/1929303838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 0x7fd00410c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 --2- 172.21.15.50:0/1929303838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004103d30 0x7fd004104140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 --2- 172.21.15.50:0/1929303838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004104e60 0x7fd004105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 -- 172.21.15.50:0/1929303838 >> 172.21.15.50:0/1929303838 conn(0x7fd0040ff560 msgr2=0x7fd004101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 -- 172.21.15.50:0/1929303838 shutdown_connections 2024-05-17T04:35:21.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 -- 172.21.15.50:0/1929303838 wait complete. 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 Processor -- start 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.076+0000 7fd009afc700 1 -- start start 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004103d30 0x7fd00419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 0x7fd0041a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410f630 con 0x7fd004104e60 2024-05-17T04:35:21.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410f4b0 con 0x7fd004103d30 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd009afc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410f7b0 con 0x7fd004105800 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50806/0 (socket says 172.21.15.50:50806) 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 -- 172.21.15.50:0/3892197388 learned_addr learned my addr 172.21.15.50:0/3892197388 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd003fff700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 0x7fd0041a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd0037fe700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004103d30 0x7fd00419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 -- 172.21.15.50:0/3892197388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 msgr2=0x7fd0041a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 0x7fd0041a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 -- 172.21.15.50:0/3892197388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004103d30 msgr2=0x7fd00419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004103d30 0x7fd00419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 -- 172.21.15.50:0/3892197388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041a1d40 con 0x7fd004104e60 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.077+0000 7fd002ffd700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d980 tx=0x7fcfec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.078+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec016070 con 0x7fd004104e60 2024-05-17T04:35:21.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.078+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfec004830 con 0x7fd004104e60 2024-05-17T04:35:21.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.078+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0041a1fd0 con 0x7fd004104e60 2024-05-17T04:35:21.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.078+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0054c0 con 0x7fd004104e60 2024-05-17T04:35:21.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.078+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0041a24b0 con 0x7fd004104e60 2024-05-17T04:35:21.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.079+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd004104140 con 0x7fd004104e60 2024-05-17T04:35:21.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.081+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcfec004220 con 0x7fd004104e60 2024-05-17T04:35:21.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.082+0000 7fd000ff9700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 0x7fcff00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:21.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.082+0000 7fd0037fe700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 0x7fcff00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:21.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.082+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcfec0e28a0 con 0x7fd004104e60 2024-05-17T04:35:21.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.083+0000 7fd0037fe700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 0x7fcff00b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009990 tx=0x7fcff4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:21.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.085+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcfec01f020 con 0x7fd004104e60 2024-05-17T04:35:21.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.307+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 --> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd004064780 con 0x7fcff00b2880 2024-05-17T04:35:21.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.309+0000 7fd000ff9700 1 -- 172.21.15.50:0/3892197388 <== mgr.14152 v2:172.21.15.50:6800/1662041432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd004064780 con 0x7fcff00b2880 2024-05-17T04:35:21.310 INFO:teuthology.orchestra.run.smithi050.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-17T04:35:21.310 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-05-17T04:35:21.310 INFO:teuthology.orchestra.run.smithi050.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-17T04:35:21.310 INFO:teuthology.orchestra.run.smithi050.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-17T04:35:21.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.312+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 msgr2=0x7fcff00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.312+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 0x7fcff00b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009990 tx=0x7fcff4005fb0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.312+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 msgr2=0x7fd00419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.312+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00d980 tx=0x7fcfec00dc90 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 shutdown_connections 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd004105800 0x7fd0041a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd004103d30 0x7fd00419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcff00b2880 0x7fcff00b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 --2- 172.21.15.50:0/3892197388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd004104e60 0x7fd00419a7b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 >> 172.21.15.50:0/3892197388 conn(0x7fd0040ff560 msgr2=0x7fd0041081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 shutdown_connections 2024-05-17T04:35:21.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:21.313+0000 7fd009afc700 1 -- 172.21.15.50:0/3892197388 wait complete. 2024-05-17T04:35:22.065 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-17T04:35:22.074 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:35:22.074 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-17T04:35:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:22 smithi050 ceph-mon[31739]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:22.331 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:22 smithi105 ceph-mon[34795]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:22.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:22 smithi110 ceph-mon[35497]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:23.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:23 smithi050 ceph-mon[31739]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:23.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:23 smithi105 ceph-mon[34795]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:23.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:23 smithi110 ceph-mon[35497]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-17T04:35:24.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.140+0000 7f26c8f52700 1 -- 172.21.15.50:0/2006340618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c4103d30 msgr2=0x7f26c4104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:24.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.140+0000 7f26c8f52700 1 --2- 172.21.15.50:0/2006340618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c4103d30 0x7f26c4104140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b0d0 tx=0x7f26ac00b3e0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 -- 172.21.15.50:0/2006340618 shutdown_connections 2024-05-17T04:35:24.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 --2- 172.21.15.50:0/2006340618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c41046d0 0x7f26c410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 --2- 172.21.15.50:0/2006340618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4102c00 0x7f26c4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 --2- 172.21.15.50:0/2006340618 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c4103d30 0x7f26c4104140 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 -- 172.21.15.50:0/2006340618 >> 172.21.15.50:0/2006340618 conn(0x7f26c40fe450 msgr2=0x7f26c41008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 -- 172.21.15.50:0/2006340618 shutdown_connections 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 -- 172.21.15.50:0/2006340618 wait complete. 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.141+0000 7f26c8f52700 1 Processor -- start 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 -- start start 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:24.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c41046d0 0x7f26c41055b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26c410e4b0 con 0x7f26c41046d0 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26c410e330 con 0x7f26c4103d30 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c8f52700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26c410e630 con 0x7f26c4102c00 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c41046d0 0x7f26c41055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c1d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59040/0 (socket says 172.21.15.50:59040) 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c1d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37276/0 (socket says 172.21.15.50:37276) 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 -- 172.21.15.50:0/1638083895 learned_addr learned my addr 172.21.15.50:0/1638083895 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 -- 172.21.15.50:0/1638083895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 msgr2=0x7f26c4105070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 -- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c41046d0 msgr2=0x7f26c41055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c41046d0 0x7f26c41055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c259c700 1 -- 172.21.15.50:0/1638083895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c4105d10 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.142+0000 7f26c1d9b700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26c259c700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00e970 tx=0x7f26bc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004a30 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26bc00cc70 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc005520 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c41a4260 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.143+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c41a46f0 con 0x7f26c4102c00 2024-05-17T04:35:24.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.144+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c4103010 con 0x7f26c4102c00 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.146+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26bc00c7e0 con 0x7f26c4102c00 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.146+0000 7f26bb7fe700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.147+0000 7f26c1d9b700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.147+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f26bc0e2930 con 0x7f26c4102c00 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.147+0000 7f26c1d9b700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 0x7f26a40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009900 tx=0x7f26ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:24.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.149+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26bc054df0 con 0x7f26c4102c00 2024-05-17T04:35:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:24 smithi050 ceph-mon[31739]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:24 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:35:24.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.421+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f26c41a53b0 con 0x7f26c4102c00 2024-05-17T04:35:24.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.424+0000 7f26bb7fe700 1 -- 172.21.15.50:0/1638083895 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3058 (secure 0 0 0) 0x7f26bc00ce20 con 0x7f26c4102c00 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:e26 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:legacy client fscid: 1 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:Filesystem 'cephfs' (1) 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:fs_name cephfs 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 26 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-05-17T04:31:45.150094+0000 2024-05-17T04:35:24.425 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-05-17T04:32:53.059994+0000 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:tableserver 0 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:root 0 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:session_timeout 300 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:session_autoclose 300 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:max_file_size 1099511627776 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:max_xattr_size 65536 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:required_client_features {} 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure 0 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:last_failure_osd_epoch 0 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:max_mds 3 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:in 0,1,2 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:up {0=24506,1=24499,2=24487} 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:failed 2024-05-17T04:35:24.426 INFO:teuthology.orchestra.run.smithi050.stdout:damaged 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:stopped 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:data_pools [3,4] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:metadata_pool 2 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:inline_data disabled 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:balancer 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:bal_rank_mask -1 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:standby_count_wanted 0 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:qdb_cluster leader: 24506 members: 24487,24506,24499 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.b{0:24506} state up:active seq 6 addr [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.h{0:24485} state up:standby-replay seq 1 addr [v2:172.21.15.105:6836/1162023031,v1:172.21.15.105:6837/1162023031] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.i{1:24499} state up:active seq 24 addr [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.f{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.110:6834/1276100312,v1:172.21.15.110:6835/1276100312] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.c{2:24487} state up:active seq 25 addr [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.427 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.e{2:24461} state up:standby-replay seq 1 addr [v2:172.21.15.105:6834/1830970686,v1:172.21.15.105:6835/1830970686] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:Standby daemons: 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.50:6834/1993874157,v1:172.21.15.50:6835/1993874157] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.50:6836/2083526209,v1:172.21.15.50:6837/2083526209] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.g{-1:14613} state up:standby seq 1 addr [v2:172.21.15.50:6838/3126760693,v1:172.21.15.50:6839/3126760693] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.50:6840/3896696320,v1:172.21.15.50:6841/3896696320] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.105:6832/32414548,v1:172.21.15.105:6833/32414548] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.428 INFO:teuthology.orchestra.run.smithi050.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.110:6832/3447375489,v1:172.21.15.110:6833/3447375489] compat {c=[1],r=[1],i=[1fff]}] 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 msgr2=0x7f26a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 0x7f26a40b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009900 tx=0x7f26ac00b040 comp rx=0 tx=0).stop 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 msgr2=0x7f26c41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00e970 tx=0x7f26bc00ec80 comp rx=0 tx=0).stop 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 shutdown_connections 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26c41046d0 0x7f26c41055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26c4102c00 0x7f26c41069c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 --2- 172.21.15.50:0/1638083895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26c4103d30 0x7f26c4105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 >> 172.21.15.50:0/1638083895 conn(0x7f26c40fe450 msgr2=0x7f26c40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 shutdown_connections 2024-05-17T04:35:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:24.427+0000 7f26c8f52700 1 -- 172.21.15.50:0/1638083895 wait complete. 2024-05-17T04:35:24.430 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:35:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:24 smithi105 ceph-mon[34795]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:24 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:35:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:24 smithi110 ceph-mon[35497]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:24 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:35:25.194 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-17T04:35:25.449 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1638083895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.755 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.756 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:25 smithi050 ceph-mon[31739]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:25.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1638083895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:25 smithi105 ceph-mon[34795]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1638083895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:25.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:25 smithi110 ceph-mon[35497]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:26.701 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:35:26.701 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:35:26.701 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:26.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:35:26.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:35:26.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:26.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:35:26.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:35:26.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 -- 172.21.15.50:0/4058531314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8103cc0 msgr2=0x7fd5e81040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/4058531314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8103cc0 0x7fd5e81040d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40099c0 tx=0x7fd5e4009cd0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 -- 172.21.15.50:0/4058531314 shutdown_connections 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/4058531314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8104660 0x7fd5e810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/4058531314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 0x7fd5e8102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/4058531314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8103cc0 0x7fd5e81040d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 -- 172.21.15.50:0/4058531314 >> 172.21.15.50:0/4058531314 conn(0x7fd5e80fe3c0 msgr2=0x7fd5e8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 -- 172.21.15.50:0/4058531314 shutdown_connections 2024-05-17T04:35:27.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.266+0000 7fd5efb2b700 1 -- 172.21.15.50:0/4058531314 wait complete. 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.267+0000 7fd5efb2b700 1 Processor -- start 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.267+0000 7fd5efb2b700 1 -- start start 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 0x7fd5e819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 0x7fd5e819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:27.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810f5f0 con 0x7fd5e8104660 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810f470 con 0x7fd5e8103cc0 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5efb2b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810f770 con 0x7fd5e8102b90 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50844/0 (socket says 172.21.15.50:50844) 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 -- 172.21.15.50:0/1904357584 learned_addr learned my addr 172.21.15.50:0/1904357584 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ed8c7700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 0x7fd5e819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ed0c6700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 0x7fd5e819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 -- 172.21.15.50:0/1904357584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 msgr2=0x7fd5e819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 0x7fd5e819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 -- 172.21.15.50:0/1904357584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 msgr2=0x7fd5e819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 0x7fd5e819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ee0c8700 1 -- 172.21.15.50:0/1904357584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e81a1cf0 con 0x7fd5e8104660 2024-05-17T04:35:27.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.268+0000 7fd5ed0c6700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 0x7fd5e819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5ee0c8700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0007d70 tx=0x7fd5e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0016020 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5e000ad40 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00115e0 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81a1fe0 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.269+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a2440 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.270+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e8102fa0 con 0x7fd5e8104660 2024-05-17T04:35:27.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.272+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd5e0019020 con 0x7fd5e8104660 2024-05-17T04:35:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.272+0000 7fd5daffd700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.272+0000 7fd5ed8c7700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:27.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.273+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd5e00e24b0 con 0x7fd5e8104660 2024-05-17T04:35:27.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.273+0000 7fd5ed8c7700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 0x7fd5d40b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005950 tx=0x7fd5dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:27.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.277+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5e0020020 con 0x7fd5e8104660 2024-05-17T04:35:27.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.495+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd5e80647a0 con 0x7fd5e8104660 2024-05-17T04:35:27.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.496+0000 7fd5daffd700 1 -- 172.21.15.50:0/1904357584 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5643 (secure 0 0 0) 0x7fd5e00a9e40 con 0x7fd5e8104660 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 89 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:created 2024-05-17T04:23:23.371105+0000 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:modified 2024-05-17T04:33:20.924661+0000 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:crush_version 27 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:full_ratio 0.95 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:backfillfull_ratio 0.9 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:nearfull_ratio 0.85 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:require_min_compat_client luminous 2024-05-17T04:35:27.496 INFO:teuthology.orchestra.run.smithi050.stdout:min_compat_client luminous 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:require_osd_release squid 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:stretch_mode_enabled false 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:max_osd 12 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6802/2573267330,v1:172.21.15.50:6803/2573267330] [v2:172.21.15.50:6804/2573267330,v1:172.21.15.50:6805/2573267330] exists,up 126dad7f-7abd-4e39-8390-c63a919ff76f 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/722687301,v1:172.21.15.105:6801/722687301] [v2:172.21.15.105:6802/722687301,v1:172.21.15.105:6803/722687301] exists,up b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/3787373914,v1:172.21.15.110:6801/3787373914] [v2:172.21.15.110:6802/3787373914,v1:172.21.15.110:6803/3787373914] exists,up ac2135de-eec8-43a3-bbeb-49c09e2b60e6 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6810/3258879203,v1:172.21.15.50:6811/3258879203] [v2:172.21.15.50:6812/3258879203,v1:172.21.15.50:6813/3258879203] exists,up 5004e0dd-7a4c-4229-abfe-72186a1ebf29 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.4 up in weight 1 up_from 31 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/908958206,v1:172.21.15.105:6809/908958206] [v2:172.21.15.105:6810/908958206,v1:172.21.15.105:6811/908958206] exists,up 2af1d73f-56c3-481a-b9ca-26dd81248baf 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.5 up in weight 1 up_from 36 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/1542934887,v1:172.21.15.110:6809/1542934887] [v2:172.21.15.110:6810/1542934887,v1:172.21.15.110:6811/1542934887] exists,up 7fec3a3b-bf8f-4065-8427-66d05c1a81a5 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6818/159814107,v1:172.21.15.50:6819/159814107] [v2:172.21.15.50:6820/159814107,v1:172.21.15.50:6821/159814107] exists,up 78770dde-1e76-4d87-b515-1127983aa2c0 2024-05-17T04:35:27.497 INFO:teuthology.orchestra.run.smithi050.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/2725719637,v1:172.21.15.105:6817/2725719637] [v2:172.21.15.105:6818/2725719637,v1:172.21.15.105:6819/2725719637] exists,up 7d035f38-f555-490f-b5bf-018aef1dbb18 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:osd.8 up in weight 1 up_from 53 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/2523141158,v1:172.21.15.110:6817/2523141158] [v2:172.21.15.110:6818/2523141158,v1:172.21.15.110:6819/2523141158] exists,up 0df6cef8-8d82-478a-98a1-980848605ac9 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6826/595367012,v1:172.21.15.50:6827/595367012] [v2:172.21.15.50:6828/595367012,v1:172.21.15.50:6829/595367012] exists,up 23c1e66f-5c60-40f9-a478-e9a4e43517ac 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/3101480579,v1:172.21.15.105:6825/3101480579] [v2:172.21.15.105:6826/3101480579,v1:172.21.15.105:6827/3101480579] exists,up d226622f-07c2-4a51-a54a-74440d67a19d 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/148686572,v1:172.21.15.110:6825/148686572] [v2:172.21.15.110:6826/148686572,v1:172.21.15.110:6827/148686572] exists,up 4ae7edc4-3891-4484-b0ca-d836265d6592 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.5 [8,11] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.c [2,1] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.f [8,11] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 2.1d [2,4] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.e [4,8] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.11 [5,9] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.13 [4,9] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 3.34 [4,9] 2024-05-17T04:35:27.498 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.e [5,8] 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.f [9,8] 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.10 [9,8] 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:pg_upmap_items 4.12 [9,10] 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/2489127888 expires 2024-05-18T04:24:18.593378+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/175302762 expires 2024-05-18T04:24:18.593378+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/2489127888 expires 2024-05-18T04:24:18.593378+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3670657062 expires 2024-05-18T04:24:18.593378+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3676423988 expires 2024-05-18T04:23:51.164359+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6801/193225580 expires 2024-05-18T04:23:51.164359+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:6800/193225580 expires 2024-05-18T04:23:51.164359+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/2016476449 expires 2024-05-18T04:24:18.593378+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3137083223 expires 2024-05-18T04:23:51.164359+0000 2024-05-17T04:35:27.499 INFO:teuthology.orchestra.run.smithi050.stdout:blocklist 172.21.15.50:0/3234552794 expires 2024-05-18T04:23:51.164359+0000 2024-05-17T04:35:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 msgr2=0x7fd5d40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 0x7fd5d40b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc005950 tx=0x7fd5dc00b410 comp rx=0 tx=0).stop 2024-05-17T04:35:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 msgr2=0x7fd5e81a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0007d70 tx=0x7fd5e000a340 comp rx=0 tx=0).stop 2024-05-17T04:35:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 shutdown_connections 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5d40b2900 0x7fd5d40b4e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e8104660 0x7fd5e81a1700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5e8102b90 0x7fd5e819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 --2- 172.21.15.50:0/1904357584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5e8103cc0 0x7fd5e819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 >> 172.21.15.50:0/1904357584 conn(0x7fd5e80fe3c0 msgr2=0x7fd5e8106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 shutdown_connections 2024-05-17T04:35:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:27.498+0000 7fd5efb2b700 1 -- 172.21.15.50:0/1904357584 wait complete. 2024-05-17T04:35:27.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:27 smithi050 ceph-mon[31739]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:27.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:27 smithi105 ceph-mon[34795]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:27.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:27 smithi110 ceph-mon[35497]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:28.219 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-17T04:35:28.231 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:35:28.489 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:28.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:28 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1904357584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T04:35:28.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:28 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1904357584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T04:35:28.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:28 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1904357584' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-17T04:35:29.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:29 smithi050 ceph-mon[31739]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:29.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:29 smithi105 ceph-mon[34795]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:29.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:29 smithi110 ceph-mon[35497]: pgmap v414: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 -- 172.21.15.50:0/2601523786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c105760 msgr2=0x7fc67c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 --2- 172.21.15.50:0/2601523786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c105760 0x7fc67c10dcb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 -- 172.21.15.50:0/2601523786 shutdown_connections 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 --2- 172.21.15.50:0/2601523786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c10e240 0x7fc67c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 --2- 172.21.15.50:0/2601523786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c104e10 0x7fc67c105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 --2- 172.21.15.50:0/2601523786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c105760 0x7fc67c10dcb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 -- 172.21.15.50:0/2601523786 >> 172.21.15.50:0/2601523786 conn(0x7fc67c079ae0 msgr2=0x7fc67c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 -- 172.21.15.50:0/2601523786 shutdown_connections 2024-05-17T04:35:30.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.216+0000 7fc680d53700 1 -- 172.21.15.50:0/2601523786 wait complete. 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.217+0000 7fc680d53700 1 Processor -- start 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.217+0000 7fc680d53700 1 -- start start 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c105760 0x7fc67c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c10e240 0x7fc67c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:30.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c113c70 con 0x7fc67c104e10 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c113af0 con 0x7fc67c10e240 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc680d53700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c113df0 con 0x7fc67c105760 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36388/0 (socket says 172.21.15.50:36388) 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 -- 172.21.15.50:0/2066994788 learned_addr learned my addr 172.21.15.50:0/2066994788 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc679d9b700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c105760 0x7fc67c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67ad9d700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c10e240 0x7fc67c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 -- 172.21.15.50:0/2066994788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c105760 msgr2=0x7fc67c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c105760 0x7fc67c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 -- 172.21.15.50:0/2066994788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c10e240 msgr2=0x7fc67c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c10e240 0x7fc67c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 -- 172.21.15.50:0/2066994788 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c19f570 con 0x7fc67c104e10 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc67a59c700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c9b0 tx=0x7fc66c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:30.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c007b00 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.218+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc66c00ce50 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.219+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c018660 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.219+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c19f860 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.219+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c19fd70 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.220+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c105220 con 0x7fc67c104e10 2024-05-17T04:35:30.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.221+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc66c00f450 con 0x7fc67c104e10 2024-05-17T04:35:30.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.222+0000 7fc66b7fe700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 0x7fc6640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:30.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.222+0000 7fc679d9b700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 0x7fc6640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:30.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.222+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc66c0e2760 con 0x7fc67c104e10 2024-05-17T04:35:30.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.223+0000 7fc679d9b700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 0x7fc6640b4e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b5c0 tx=0x7fc670005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:30.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.225+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc66c014030 con 0x7fc67c104e10 2024-05-17T04:35:30.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.485+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67c02ea80 con 0x7fc67c104e10 2024-05-17T04:35:30.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.486+0000 7fc66b7fe700 1 -- 172.21.15.50:0/2066994788 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7fc66c0aa1a0 con 0x7fc67c104e10 2024-05-17T04:35:30.487 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:30.487 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:35:30.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 msgr2=0x7fc6640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:30.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 0x7fc6640b4e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b5c0 tx=0x7fc670005960 comp rx=0 tx=0).stop 2024-05-17T04:35:30.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 msgr2=0x7fc67c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c9b0 tx=0x7fc66c00ccc0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 shutdown_connections 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc67c10e240 0x7fc67c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc6640b2900 0x7fc6640b4e10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc67c104e10 0x7fc67c19a750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 --2- 172.21.15.50:0/2066994788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc67c105760 0x7fc67c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 >> 172.21.15.50:0/2066994788 conn(0x7fc67c079ae0 msgr2=0x7fc67c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 shutdown_connections 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:30.488+0000 7fc680d53700 1 -- 172.21.15.50:0/2066994788 wait complete. 2024-05-17T04:35:30.490 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:35:30.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:30 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2066994788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:30.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:30 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2066994788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:30.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:30 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2066994788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:31.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:35:31.464 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:31.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:31 smithi105 ceph-mon[34795]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:31.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:31 smithi110 ceph-mon[35497]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:31.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:31 smithi050 ceph-mon[31739]: pgmap v415: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:33.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 -- 172.21.15.50:0/1615860419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8105790 msgr2=0x7f4ce810dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:33.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 --2- 172.21.15.50:0/1615860419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8105790 0x7f4ce810dce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009990 tx=0x7f4cdc009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 -- 172.21.15.50:0/1615860419 shutdown_connections 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 --2- 172.21.15.50:0/1615860419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce810e220 0x7f4ce8110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 --2- 172.21.15.50:0/1615860419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce8104e40 0x7f4ce8105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 --2- 172.21.15.50:0/1615860419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8105790 0x7f4ce810dce0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 -- 172.21.15.50:0/1615860419 >> 172.21.15.50:0/1615860419 conn(0x7f4ce8079bd0 msgr2=0x7f4ce8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 -- 172.21.15.50:0/1615860419 shutdown_connections 2024-05-17T04:35:33.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.190+0000 7f4cf064c700 1 -- 172.21.15.50:0/1615860419 wait complete. 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.191+0000 7f4cf064c700 1 Processor -- start 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.191+0000 7f4cf064c700 1 -- start start 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.191+0000 7f4cf064c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.191+0000 7f4cf064c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8105790 0x7f4ce8103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.191+0000 7f4cf064c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce810e220 0x7f4ce81035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cf064c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8113c50 con 0x7f4ce8104e40 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cf064c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8113ad0 con 0x7f4ce8105790 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cf064c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8113dd0 con 0x7f4ce810e220 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36408/0 (socket says 172.21.15.50:36408) 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 -- 172.21.15.50:0/2596389018 learned_addr learned my addr 172.21.15.50:0/2596389018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:33.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cedbe7700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8105790 0x7f4ce8103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4ceebe9700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce810e220 0x7f4ce81035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 -- 172.21.15.50:0/2596389018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce810e220 msgr2=0x7f4ce81035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce810e220 0x7f4ce81035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 -- 172.21.15.50:0/2596389018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8105790 msgr2=0x7f4ce8103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8105790 0x7f4ce8103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 -- 172.21.15.50:0/2596389018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce81a8ab0 con 0x7f4ce8104e40 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cee3e8700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ada0 tx=0x7f4cd800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8017020 con 0x7f4ce8104e40 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cd800ce10 con 0x7f4ce8104e40 2024-05-17T04:35:33.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.192+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce81a8da0 con 0x7f4ce8104e40 2024-05-17T04:35:33.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.193+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a9250 con 0x7f4ce8104e40 2024-05-17T04:35:33.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.193+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8005310 con 0x7f4ce8104e40 2024-05-17T04:35:33.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.193+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce8105250 con 0x7f4ce8104e40 2024-05-17T04:35:33.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.195+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4cd8004ad0 con 0x7f4ce8104e40 2024-05-17T04:35:33.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.196+0000 7f4ce77fe700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 0x7f4cd00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:33.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.196+0000 7f4cedbe7700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 0x7f4cd00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:33.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.196+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4cd80e2430 con 0x7f4ce8104e40 2024-05-17T04:35:33.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.197+0000 7f4cedbe7700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 0x7f4cd00b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0098e0 tx=0x7f4cdc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:33.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.199+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cd8021020 con 0x7f4ce8104e40 2024-05-17T04:35:33.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.460+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ce8104060 con 0x7f4ce8104e40 2024-05-17T04:35:33.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.461+0000 7f4ce77fe700 1 -- 172.21.15.50:0/2596389018 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11882 (secure 0 0 0) 0x7f4cd80a9df0 con 0x7f4ce8104e40 2024-05-17T04:35:33.462 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:33.462 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:32:53.059994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24506,24499]},"id":1}]} 2024-05-17T04:35:33.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 msgr2=0x7f4cd00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:33.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 0x7f4cd00b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0098e0 tx=0x7f4cdc005960 comp rx=0 tx=0).stop 2024-05-17T04:35:33.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 msgr2=0x7f4ce8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800ada0 tx=0x7f4cd800c330 comp rx=0 tx=0).stop 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 shutdown_connections 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ce810e220 0x7f4ce81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4cd00b2900 0x7f4cd00b4e10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ce8104e40 0x7f4ce8102b30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 --2- 172.21.15.50:0/2596389018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce8105790 0x7f4ce8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 >> 172.21.15.50:0/2596389018 conn(0x7f4ce8079bd0 msgr2=0x7f4ce80ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 shutdown_connections 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:33.463+0000 7f4cf064c700 1 -- 172.21.15.50:0/2596389018 wait complete. 2024-05-17T04:35:33.465 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 26 2024-05-17T04:35:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:33 smithi050 ceph-mon[31739]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:34.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:33 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2596389018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:34.169 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:35:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:33 smithi105 ceph-mon[34795]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:33 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2596389018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:33 smithi110 ceph-mon[35497]: pgmap v416: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:33 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2596389018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:35:34.423 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:35.876 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:35 smithi050 ceph-mon[31739]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 -- 172.21.15.50:0/2751584328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c106180 msgr2=0x7fa31c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 --2- 172.21.15.50:0/2751584328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c106180 0x7fa31c110bd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa31400b0a0 tx=0x7fa31400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 -- 172.21.15.50:0/2751584328 shutdown_connections 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 --2- 172.21.15.50:0/2751584328 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c106180 0x7fa31c110bd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 --2- 172.21.15.50:0/2751584328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c104010 0x7fa31c104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 --2- 172.21.15.50:0/2751584328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c105830 0x7fa31c105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 -- 172.21.15.50:0/2751584328 >> 172.21.15.50:0/2751584328 conn(0x7fa31c0ff9b0 msgr2=0x7fa31c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 -- 172.21.15.50:0/2751584328 shutdown_connections 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.303+0000 7fa3226e7700 1 -- 172.21.15.50:0/2751584328 wait complete. 2024-05-17T04:35:36.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 Processor -- start 2024-05-17T04:35:36.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 -- start start 2024-05-17T04:35:36.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 0x7fa31c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:36.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c106180 0x7fa31c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113d90 con 0x7fa31c105830 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113c10 con 0x7fa31c104010 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa3226e7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113f10 con 0x7fa31c106180 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa31b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa31bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 0x7fa31c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.304+0000 7fa31b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36418/0 (socket says 172.21.15.50:36418) 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 0x7fa31c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37214/0 (socket says 172.21.15.50:37214) 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 -- 172.21.15.50:0/3733565003 learned_addr learned my addr 172.21.15.50:0/3733565003 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa320c84700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c106180 0x7fa31c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 -- 172.21.15.50:0/3733565003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c106180 msgr2=0x7fa31c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c106180 0x7fa31c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 -- 172.21.15.50:0/3733565003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 msgr2=0x7fa31c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:36.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 0x7fa31c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 -- 172.21.15.50:0/3733565003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c19f600 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa31b7fe700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d950 tx=0x7fa31000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310016070 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c19f890 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa310004830 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310005420 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.305+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a7210 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.307+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31c104420 con 0x7fa31c105830 2024-05-17T04:35:36.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.308+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3100055e0 con 0x7fa31c105830 2024-05-17T04:35:36.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.309+0000 7fa3197fa700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 0x7fa3040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:36.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.309+0000 7fa31bfff700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 0x7fa3040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:36.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.309+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa3100e2310 con 0x7fa31c105830 2024-05-17T04:35:36.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.310+0000 7fa31bfff700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 0x7fa3040b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005950 tx=0x7fa30c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:35 smithi105 ceph-mon[34795]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:36.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.314+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa31001f020 con 0x7fa31c105830 2024-05-17T04:35:36.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:35 smithi110 ceph-mon[35497]: pgmap v417: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:36.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.531+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa31c19ba90 con 0x7fa31c105830 2024-05-17T04:35:36.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.533+0000 7fa3197fa700 1 -- 172.21.15.50:0/3733565003 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7fa3100a9c40 con 0x7fa31c105830 2024-05-17T04:35:36.535 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:36.535 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:35:36.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 msgr2=0x7fa3040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 0x7fa3040b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005950 tx=0x7fa30c00a300 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 msgr2=0x7fa31c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d950 tx=0x7fa31000dc60 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 shutdown_connections 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa31c106180 0x7fa31c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa31c104010 0x7fa31c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa3040b2880 0x7fa3040b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 --2- 172.21.15.50:0/3733565003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa31c105830 0x7fa31c19acd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.536+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 >> 172.21.15.50:0/3733565003 conn(0x7fa31c0ff9b0 msgr2=0x7fa31c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.537+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 shutdown_connections 2024-05-17T04:35:36.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:36.537+0000 7fa3226e7700 1 -- 172.21.15.50:0/3733565003 wait complete. 2024-05-17T04:35:37.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:36 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3733565003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:35:37.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:36 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3733565003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:35:37.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:36 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3733565003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:35:37.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-17T04:31:45.150094+0000', 'modified': '2024-05-17T04:32:53.059994+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24499, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/1830970686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1830970686}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1830970686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/1276100312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 1276100312}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 1276100312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/1162023031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 1162023031}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 1162023031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.110:6837/2935546223', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2935546223}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2935546223}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.110:6839/2082916819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 2082916819}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 2082916819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/3722840784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 3722840784}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 3722840784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24487, 24506, 24499]} 2024-05-17T04:35:37.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-17T04:35:37.384 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-17T04:35:37.641 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:38.034 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:37 smithi050 ceph-mon[31739]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:38.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:37 smithi105 ceph-mon[34795]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:37 smithi110 ceph-mon[35497]: pgmap v418: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:39.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 -- 172.21.15.50:0/4037166931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c1080a0 msgr2=0x7fbf0c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 --2- 172.21.15.50:0/4037166931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c10fa00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b0a0 tx=0x7fbf0400b3b0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 -- 172.21.15.50:0/4037166931 shutdown_connections 2024-05-17T04:35:39.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 --2- 172.21.15.50:0/4037166931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c10fa00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 --2- 172.21.15.50:0/4037166931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 --2- 172.21.15.50:0/4037166931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c107750 0x7fbf0c107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 -- 172.21.15.50:0/4037166931 >> 172.21.15.50:0/4037166931 conn(0x7fbf0c079bd0 msgr2=0x7fbf0c079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 -- 172.21.15.50:0/4037166931 shutdown_connections 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.490+0000 7fbf116c9700 1 -- 172.21.15.50:0/4037166931 wait complete. 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 Processor -- start 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 -- start start 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c1a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112b90 con 0x7fbf0c107750 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112a10 con 0x7fbf0c102e20 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf116c9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c112d10 con 0x7fbf0c1080a0 2024-05-17T04:35:39.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50996/0 (socket says 172.21.15.50:50996) 2024-05-17T04:35:39.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.491+0000 7fbf03fff700 1 -- 172.21.15.50:0/2929492447 learned_addr learned my addr 172.21.15.50:0/2929492447 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c1a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 -- 172.21.15.50:0/2929492447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c1080a0 msgr2=0x7fbf0c1a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c1a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 -- 172.21.15.50:0/2929492447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 msgr2=0x7fbf0c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 -- 172.21.15.50:0/2929492447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c19c1a0 con 0x7fbf0c107750 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf03fff700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c9e0 tx=0x7fbef400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf08ff9700 1 -- 172.21.15.50:0/2929492447 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4007aa0 con 0x7fbf0c107750 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1a6a20 con 0x7fbf0c107750 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf08ff9700 1 -- 172.21.15.50:0/2929492447 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef4007c40 con 0x7fbf0c107750 2024-05-17T04:35:39.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf08ff9700 1 -- 172.21.15.50:0/2929492447 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4018660 con 0x7fbf0c107750 2024-05-17T04:35:39.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.492+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a6f80 con 0x7fbf0c107750 2024-05-17T04:35:39.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.495+0000 7fbf08ff9700 1 -- 172.21.15.50:0/2929492447 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbef400f450 con 0x7fbf0c107750 2024-05-17T04:35:39.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.496+0000 7fbf08ff9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 0x7fbef80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.496+0000 7fbf200a4b80 1 Processor -- start 2024-05-17T04:35:39.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.496+0000 7fbf200a4b80 1 -- start start 2024-05-17T04:35:39.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.496+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 0x7fbef80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.496+0000 7fbf08ff9700 1 -- 172.21.15.50:0/2929492447 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbef40e2500 con 0x7fbf0c107750 2024-05-17T04:35:39.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 0x7fbef80b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00bc10 tx=0x7fbefc00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c1402510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c8c132b240 con 0x55c8c13fd120 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8c131af80 con 0x55c8c127e9a0 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c8c131ee20 con 0x55c8c1402a50 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51008/0 (socket says 172.21.15.50:51008) 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/724597310 learned_addr learned my addr 172.21.15.50:0/724597310 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0affd700 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c1402510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf03fff700 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/724597310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 msgr2=0x55c8c13fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/724597310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 msgr2=0x55c8c1402510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c1402510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.497+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/724597310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c12c4540 con 0x55c8c13fd120 2024-05-17T04:35:39.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.498+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbf04015000 tx=0x7fbf04009150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8214ce1dbf7efe00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.498+0000 7fbf017fa700 1 -- 172.21.15.50:0/724597310 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04020070 con 0x55c8c13fd120 2024-05-17T04:35:39.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.498+0000 7fbf017fa700 1 -- 172.21.15.50:0/724597310 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf040075c0 con 0x55c8c13fd120 2024-05-17T04:35:39.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.498+0000 7fbf017fa700 1 -- 172.21.15.50:0/724597310 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040052a0 con 0x55c8c13fd120 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- 172.21.15.50:0/724597310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 msgr2=0x55c8c13fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbf04015000 tx=0x7fbf04009150 comp rx=0 tx=0).stop 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- 172.21.15.50:0/724597310 shutdown_connections 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13fd530 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c1402510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/724597310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- 172.21.15.50:0/724597310 >> 172.21.15.50:0/724597310 conn(0x55c8c12c2eb0 msgr2=0x55c8c13f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- 172.21.15.50:0/724597310 shutdown_connections 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- 172.21.15.50:0/724597310 wait complete. 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.499+0000 7fbf200a4b80 1 -- start start 2024-05-17T04:35:39.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 Processor -- start 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 0x55c8c14061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1222100 con 0x55c8c1402a50 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8c12bbe30 con 0x55c8c13fd120 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1409340 con 0x55c8c127e9a0 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf200a4b80 10 client.0 _ll_register_callbacks cb 0x55c8c12b5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf0b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf0b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51012/0 (socket says 172.21.15.50:51012) 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/2279199964 learned_addr learned my addr 172.21.15.50:0/2279199964 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38232/0 (socket says 172.21.15.50:38232) 2024-05-17T04:35:39.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.500+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 0x55c8c14061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 msgr2=0x55c8c14061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 0x55c8c14061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/2279199964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 msgr2=0x55c8c1406730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 -- 172.21.15.50:0/2279199964 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c126b430 con 0x55c8c1402a50 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf03fff700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 0x55c8c14061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009c10 tx=0x7fbf040095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50aa828a9251248 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbef27fc700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04020040 con 0x55c8c1402a50 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbf04005040 con 0x55c8c1402a50 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf040075c0 con 0x55c8c1402a50 2024-05-17T04:35:39.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf04005240 con 0x55c8c1402a50 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf200a4b80 10 client.15189 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8c1413dd0 con 0x55c8c1402a50 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.501+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf040075c0 con 0x55c8c1402a50 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbf04011070 con 0x55c8c1402a50 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 10 client.15189 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 20 client.15189 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1415120 0x55c8c14172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 4 client.15189 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 --> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c132d190 con 0x55c8c1415120 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.502+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1415120 0x55c8c14172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.503+0000 7fbef27fc700 10 client.15189 ms_handle_connect on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:39.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.503+0000 7fbf0affd700 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1415120 0x55c8c14172c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9483871a3c2a010b server_cookie=a4937289863e2e40 in_seq=0 out_seq=0 2024-05-17T04:35:39.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.503+0000 7fbef27fc700 1 -- 172.21.15.50:0/2279199964 <== mds.2 v2:172.21.15.110:6836/2935546223 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8c132d190 con 0x55c8c1415120 2024-05-17T04:35:39.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.503+0000 7fbef27fc700 10 client.15189 handle_command_reply: tid=0 2024-05-17T04:35:39.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 client.15189 shutdown 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 msgr2=0x55c8c1406cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009c10 tx=0x7fbf040095a0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 shutdown_connections 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c1402a50 0x55c8c1406cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c127e9a0 0x55c8c14061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1415120 0x55c8c14172c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/2279199964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c13fd120 0x55c8c1406730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbef27fc700 0 client.15189 ms_handle_reset on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 >> 172.21.15.50:0/2279199964 conn(0x55c8c12c2eb0 msgr2=0x55c8c13f1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 shutdown_connections 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 1 -- 172.21.15.50:0/2279199964 wait complete. 2024-05-17T04:35:39.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.504+0000 7fbf200a4b80 20 client.15189 trim_cache size 0 max 16384 2024-05-17T04:35:39.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 Processor -- start 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 -- start start 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c13fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1415600 con 0x55c8c13fd120 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1415cd0 con 0x55c8c127e9a0 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c8c14163a0 con 0x55c8c1402a50 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51014/0 (socket says 172.21.15.50:51014) 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 -- 172.21.15.50:0/1216879388 learned_addr learned my addr 172.21.15.50:0/1216879388 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf0affd700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c13fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 -- 172.21.15.50:0/1216879388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 msgr2=0x55c8c13f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 -- 172.21.15.50:0/1216879388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 msgr2=0x55c8c13fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c13fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.526+0000 7fbf03fff700 1 -- 172.21.15.50:0/1216879388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c14024f0 con 0x55c8c13fd120 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.527+0000 7fbf0affd700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c13fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.527+0000 7fbf03fff700 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbef40dfc10 tx=0x7fbef40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98388bae6fd513a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.527+0000 7fbf00ff9700 1 -- 172.21.15.50:0/1216879388 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b0e60 con 0x55c8c13fd120 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.527+0000 7fbf00ff9700 1 -- 172.21.15.50:0/1216879388 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef40a3e50 con 0x55c8c13fd120 2024-05-17T04:35:39.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.527+0000 7fbf00ff9700 1 -- 172.21.15.50:0/1216879388 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b2360 con 0x55c8c13fd120 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- 172.21.15.50:0/1216879388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 msgr2=0x55c8c13f1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbef40dfc10 tx=0x7fbef40cc040 comp rx=0 tx=0).stop 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- 172.21.15.50:0/1216879388 shutdown_connections 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c1402a50 0x55c8c13f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c127e9a0 0x55c8c13fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/1216879388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c13fd120 0x55c8c13f1e60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- 172.21.15.50:0/1216879388 >> 172.21.15.50:0/1216879388 conn(0x55c8c12c2eb0 msgr2=0x55c8c13f7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- 172.21.15.50:0/1216879388 shutdown_connections 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- 172.21.15.50:0/1216879388 wait complete. 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 -- start start 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.528+0000 7fbf200a4b80 1 Processor -- start 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 0x55c8c141bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 0x55c8c141c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1417140 con 0x55c8c127e9a0 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1416a70 con 0x55c8c1402a50 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1417810 con 0x55c8c13fd120 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf200a4b80 10 client.0 _ll_register_callbacks cb 0x55c8c12b5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51028/0 (socket says 172.21.15.50:51028) 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 -- 172.21.15.50:0/3287849459 learned_addr learned my addr 172.21.15.50:0/3287849459 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:39.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf03fff700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 0x55c8c141bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 0x55c8c141c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 -- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 msgr2=0x55c8c141bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 0x55c8c141bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 -- 172.21.15.50:0/3287849459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 msgr2=0x55c8c141c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 0x55c8c141c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 -- 172.21.15.50:0/3287849459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c141c730 con 0x55c8c127e9a0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf03fff700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 0x55c8c141bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0b7fe700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 0x55c8c141c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbf0affd700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55c8c13f8a90 tx=0x7fbefc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b09d4fe58d7310c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.529+0000 7fbef27fc700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc028490 con 0x55c8c127e9a0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbefc028630 con 0x55c8c127e9a0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef40a3e50 con 0x55c8c127e9a0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbefc03a050 con 0x55c8c127e9a0 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 10 client.15195 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:39.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8c1480f40 con 0x55c8c127e9a0 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef40a3e50 con 0x55c8c127e9a0 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbefc01fb30 con 0x55c8c127e9a0 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 20 client.15195 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1483490 0x55c8c1484900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 4 client.15195 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.530+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 --> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] -- command(tid 0: {"prefix": "status"}) -- 0x55c8c1326970 con 0x55c8c1483490 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.531+0000 7fbf03fff700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1483490 0x55c8c1484900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.531+0000 7fbf03fff700 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1483490 0x55c8c1484900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=550cd90ba8c8fda9 server_cookie=1746ae6fe5b5eeb in_seq=0 out_seq=0 2024-05-17T04:35:39.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.531+0000 7fbef27fc700 10 client.15195 ms_handle_connect on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbef27fc700 1 -- 172.21.15.50:0/3287849459 <== mds.2 v2:172.21.15.110:6836/2935546223 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c8c1326970 con 0x55c8c1483490 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbef27fc700 10 client.15195 handle_command_reply: tid=0 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 client.15195 shutdown 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 msgr2=0x55c8c1420950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55c8c13f8a90 tx=0x7fbefc0047c0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 shutdown_connections 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:6836/2935546223,v1:172.21.15.110:6837/2935546223] conn(0x55c8c1483490 0x55c8c1484900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c1402a50 0x55c8c141c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c8c127e9a0 0x55c8c1420950 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 --2- 172.21.15.50:0/3287849459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c8c13fd120 0x55c8c141bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 >> 172.21.15.50:0/3287849459 conn(0x55c8c12c2eb0 msgr2=0x55c8c13f7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbef27fc700 0 client.15195 ms_handle_reset on v2:172.21.15.110:6836/2935546223 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 shutdown_connections 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 1 -- 172.21.15.50:0/3287849459 wait complete. 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.532+0000 7fbf200a4b80 20 client.15195 trim_cache size 0 max 16384 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:39.536 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 2, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24487, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 171.50063356000001, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 266.92628936699998 2024-05-17T04:35:39.537 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 msgr2=0x7fbef80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 0x7fbef80b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00bc10 tx=0x7fbefc00b5f0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 msgr2=0x7fbf0c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c9e0 tx=0x7fbef400ccf0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 shutdown_connections 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf0c1080a0 0x7fbf0c1a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf0c102e20 0x7fbf0c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbef80b2880 0x7fbef80b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 --2- 172.21.15.50:0/2929492447 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf0c107750 0x7fbf0c19b9a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:39.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 >> 172.21.15.50:0/2929492447 conn(0x7fbf0c079bd0 msgr2=0x7fbf0c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:39.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 shutdown_connections 2024-05-17T04:35:39.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:39.533+0000 7fbf116c9700 1 -- 172.21.15.50:0/2929492447 wait complete. 2024-05-17T04:35:40.111 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:39 smithi050 ceph-mon[31739]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:39 smithi105 ceph-mon[34795]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:40.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:39 smithi110 ceph-mon[35497]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:40.403 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-17T04:35:40.657 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:42.115 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:41 smithi050 ceph-mon[31739]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:41 smithi105 ceph-mon[34795]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:41 smithi110 ceph-mon[35497]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.551+0000 7fb829e73700 1 -- 172.21.15.50:0/4005108898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb824102e70 msgr2=0x7fb824103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.551+0000 7fb829e73700 1 --2- 172.21.15.50:0/4005108898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb824102e70 0x7fb824103280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb814009990 tx=0x7fb814009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 -- 172.21.15.50:0/4005108898 shutdown_connections 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 --2- 172.21.15.50:0/4005108898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241080a0 0x7fb82410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 --2- 172.21.15.50:0/4005108898 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb824102e70 0x7fb824103280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 --2- 172.21.15.50:0/4005108898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 0x7fb824107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 -- 172.21.15.50:0/4005108898 >> 172.21.15.50:0/4005108898 conn(0x7fb824079ae0 msgr2=0x7fb824079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 -- 172.21.15.50:0/4005108898 shutdown_connections 2024-05-17T04:35:42.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 -- 172.21.15.50:0/4005108898 wait complete. 2024-05-17T04:35:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 Processor -- start 2024-05-17T04:35:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.552+0000 7fb829e73700 1 -- start start 2024-05-17T04:35:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824102e70 0x7fb82419b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 0x7fb82419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb824112b80 con 0x7fb8241080a0 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb824112a00 con 0x7fb824102e70 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb829e73700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb824112d00 con 0x7fb824107750 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51036/0 (socket says 172.21.15.50:51036) 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 -- 172.21.15.50:0/546839314 learned_addr learned my addr 172.21.15.50:0/546839314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb81affd700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 0x7fb82419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb8237fe700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824102e70 0x7fb82419b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 -- 172.21.15.50:0/546839314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 msgr2=0x7fb82419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 0x7fb82419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 -- 172.21.15.50:0/546839314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824102e70 msgr2=0x7fb82419b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824102e70 0x7fb82419b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.553+0000 7fb823fff700 1 -- 172.21.15.50:0/546839314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8241a6830 con 0x7fb8241080a0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb823fff700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb81c007d70 tx=0x7fb81c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb8217fa700 1 -- 172.21.15.50:0/546839314 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c016020 con 0x7fb8241080a0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8241a6b20 con 0x7fb8241080a0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb8217fa700 1 -- 172.21.15.50:0/546839314 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb81c00ac90 con 0x7fb8241080a0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb8217fa700 1 -- 172.21.15.50:0/546839314 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0115e0 con 0x7fb8241080a0 2024-05-17T04:35:42.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.554+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a7010 con 0x7fb8241080a0 2024-05-17T04:35:42.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.557+0000 7fb8217fa700 1 -- 172.21.15.50:0/546839314 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb81c019020 con 0x7fb8241080a0 2024-05-17T04:35:42.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb8217fa700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 0x7fb8100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb8237fe700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 0x7fb8100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb8217fa700 1 -- 172.21.15.50:0/546839314 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb81c0e24f0 con 0x7fb8241080a0 2024-05-17T04:35:42.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb8237fe700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 0x7fb8100b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b5c0 tx=0x7fb814005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb83884eb80 1 Processor -- start 2024-05-17T04:35:42.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.558+0000 7fb83884eb80 1 -- start start 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56124bee5240 con 0x56124bfbca50 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56124bed4f80 con 0x56124bfb7120 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb83884eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56124bed8e20 con 0x56124be389a0 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51040/0 (socket says 172.21.15.50:51040) 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 -- 172.21.15.50:0/1582795349 learned_addr learned my addr 172.21.15.50:0/1582795349 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb8237fe700 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb823fff700 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 -- 172.21.15.50:0/1582795349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 msgr2=0x56124bfbc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 -- 172.21.15.50:0/1582795349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 msgr2=0x56124bfb7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfb7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 -- 172.21.15.50:0/1582795349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124be7e540 con 0x56124bfbca50 2024-05-17T04:35:42.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb81affd700 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00bcf0 tx=0x7fb80c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d303486ead06e460 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.559+0000 7fb819ffb700 1 -- 172.21.15.50:0/1582795349 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c010dc0 con 0x56124bfbca50 2024-05-17T04:35:42.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb819ffb700 1 -- 172.21.15.50:0/1582795349 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb80c0044c0 con 0x56124bfbca50 2024-05-17T04:35:42.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb819ffb700 1 -- 172.21.15.50:0/1582795349 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c005280 con 0x56124bfbca50 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 -- 172.21.15.50:0/1582795349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 msgr2=0x56124bfb6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00bcf0 tx=0x7fb80c0103d0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 -- 172.21.15.50:0/1582795349 shutdown_connections 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfb7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 --2- 172.21.15.50:0/1582795349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfbca50 0x56124bfb6be0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.560+0000 7fb83884eb80 1 -- 172.21.15.50:0/1582795349 >> 172.21.15.50:0/1582795349 conn(0x56124be7ceb0 msgr2=0x56124bfb1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.561+0000 7fb83884eb80 1 -- 172.21.15.50:0/1582795349 shutdown_connections 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.561+0000 7fb83884eb80 1 -- 172.21.15.50:0/1582795349 wait complete. 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.561+0000 7fb83884eb80 1 -- start start 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.561+0000 7fb83884eb80 1 Processor -- start 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfc0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 0x56124bfc0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56124bddc100 con 0x56124be389a0 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56124be75e30 con 0x56124bfb7120 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56124bfc3340 con 0x56124bfbca50 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 10 client.0 _ll_register_callbacks cb 0x56124be6fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51056/0 (socket says 172.21.15.50:51056) 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 -- 172.21.15.50:0/2300965504 learned_addr learned my addr 172.21.15.50:0/2300965504 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb81affd700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfc0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb823fff700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 0x56124bfc0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 -- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 msgr2=0x56124bfc0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 0x56124bfc0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 -- 172.21.15.50:0/2300965504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 msgr2=0x56124bfc0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfc0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 -- 172.21.15.50:0/2300965504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124be25430 con 0x56124be389a0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb823fff700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 0x56124bfc0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb81affd700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfc0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c00 tx=0x7fb814009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3efb99a2f7ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814034060 con 0x56124be389a0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80c0044c0 con 0x56124be389a0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.562+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb81400f460 con 0x56124be389a0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb81403b050 con 0x56124be389a0 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 10 client.15207 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:42.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56124bfcdda0 con 0x56124be389a0 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb80c0044c0 con 0x56124be389a0 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb81402e070 con 0x56124be389a0 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 10 client.15207 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 20 client.15207 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124bfd0460 0x56124bfd18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 4 client.15207 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 --> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56124bee7190 con 0x56124bfd0460 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.563+0000 7fb81affd700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124bfd0460 0x56124bfd18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.564+0000 7fb81affd700 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124bfd0460 0x56124bfd18d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c77c5e264e2696a server_cookie=e54c1c183a656c9 in_seq=0 out_seq=0 2024-05-17T04:35:42.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.564+0000 7fb802ffd700 10 client.15207 ms_handle_connect on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:42.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb802ffd700 1 -- 172.21.15.50:0/2300965504 <== mds.1 v2:172.21.15.110:6838/2082916819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56124bee7190 con 0x56124bfd0460 2024-05-17T04:35:42.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb802ffd700 10 client.15207 handle_command_reply: tid=0 2024-05-17T04:35:42.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 client.15207 shutdown 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 msgr2=0x56124bfc0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c00 tx=0x7fb814009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 shutdown_connections 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124bfd0460 0x56124bfd18d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124bfbca50 0x56124bfc0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb802ffd700 0 client.15207 ms_handle_reset on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124be389a0 0x56124bfc0140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2300965504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfb7120 0x56124bfc0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 >> 172.21.15.50:0/2300965504 conn(0x56124be7ceb0 msgr2=0x56124bfabe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 shutdown_connections 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 1 -- 172.21.15.50:0/2300965504 wait complete. 2024-05-17T04:35:42.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.565+0000 7fb83884eb80 20 client.15207 trim_cache size 0 max 16384 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 Processor -- start 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 -- start start 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfb6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56124bfcfc10 con 0x56124bfb7120 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56124bfd02e0 con 0x56124bfbca50 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb83884eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56124bfd09b0 con 0x56124be389a0 2024-05-17T04:35:42.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51068/0 (socket says 172.21.15.50:51068) 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.587+0000 7fb81affd700 1 -- 172.21.15.50:0/3598102816 learned_addr learned my addr 172.21.15.50:0/3598102816 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb823fff700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 -- 172.21.15.50:0/3598102816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 msgr2=0x56124bfb6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb8237fe700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfb6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfb6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 -- 172.21.15.50:0/3598102816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 msgr2=0x56124bfc1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 -- 172.21.15.50:0/3598102816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124bfbe690 con 0x56124bfb7120 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb823fff700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb81affd700 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56124bfbbfe0 tx=0x7fb80c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3511a6ab5f8d6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb8197fa700 1 -- 172.21.15.50:0/3598102816 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004270 con 0x56124bfb7120 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb8197fa700 1 -- 172.21.15.50:0/3598102816 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb80c004410 con 0x56124bfb7120 2024-05-17T04:35:42.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.588+0000 7fb8197fa700 1 -- 172.21.15.50:0/3598102816 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0116a0 con 0x56124bfb7120 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 -- 172.21.15.50:0/3598102816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 msgr2=0x56124bfabe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56124bfbbfe0 tx=0x7fb80c010d50 comp rx=0 tx=0).stop 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 -- 172.21.15.50:0/3598102816 shutdown_connections 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfb6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 --2- 172.21.15.50:0/3598102816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfabe60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 -- 172.21.15.50:0/3598102816 >> 172.21.15.50:0/3598102816 conn(0x56124be7ceb0 msgr2=0x56124bfb1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 -- 172.21.15.50:0/3598102816 shutdown_connections 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.589+0000 7fb83884eb80 1 -- 172.21.15.50:0/3598102816 wait complete. 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 -- start start 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 Processor -- start 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56124bfd1750 con 0x56124bfb7120 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56124bfd1080 con 0x56124bfbca50 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56124bfd1e20 con 0x56124be389a0 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb83884eb80 10 client.0 _ll_register_callbacks cb 0x56124be6fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:42.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb81affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51074/0 (socket says 172.21.15.50:51074) 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb81affd700 1 -- 172.21.15.50:0/2949166252 learned_addr learned my addr 172.21.15.50:0/2949166252 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb823fff700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.590+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 -- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 msgr2=0x56124bfbffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 -- 172.21.15.50:0/2949166252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 msgr2=0x56124bfc0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 -- 172.21.15.50:0/2949166252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124bfdb790 con 0x56124bfb7120 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb823fff700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb81affd700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00bc10 tx=0x7fb80c028430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71d55b8315b9dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0289a0 con 0x56124bfb7120 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb80c011440 con 0x56124bfb7120 2024-05-17T04:35:42.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80c004410 con 0x56124bfb7120 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb80c03a6d0 con 0x56124bfb7120 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb83884eb80 10 client.15213 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56124c03b990 con 0x56124bfb7120 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.591+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb80c004410 con 0x56124bfb7120 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb80c031420 con 0x56124bfb7120 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 10 client.15213 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 20 client.15213 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124c03d6f0 0x56124c03eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 4 client.15213 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 --> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] -- command(tid 0: {"prefix": "status"}) -- 0x56124bee0970 con 0x56124c03d6f0 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124c03d6f0 0x56124c03eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb8237fe700 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124c03d6f0 0x56124c03eb60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2857e792a51e268 server_cookie=2488b74985504085 in_seq=0 out_seq=0 2024-05-17T04:35:42.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.592+0000 7fb802ffd700 10 client.15213 ms_handle_connect on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb802ffd700 1 -- 172.21.15.50:0/2949166252 <== mds.1 v2:172.21.15.110:6838/2082916819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56124bee0970 con 0x56124c03d6f0 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb802ffd700 10 client.15213 handle_command_reply: tid=0 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 client.15213 shutdown 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 msgr2=0x56124bfc0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00bc10 tx=0x7fb80c028430 comp rx=0 tx=0).stop 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 shutdown_connections 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:6838/2082916819,v1:172.21.15.110:6839/2082916819] conn(0x56124c03d6f0 0x56124c03eb60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56124bfbca50 0x56124bfc0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56124be389a0 0x56124bfbffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 --2- 172.21.15.50:0/2949166252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56124bfb7120 0x56124bfc0500 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 >> 172.21.15.50:0/2949166252 conn(0x56124be7ceb0 msgr2=0x56124bfb1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb802ffd700 0 client.15213 ms_handle_reset on v2:172.21.15.110:6838/2082916819 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 shutdown_connections 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 1 -- 172.21.15.50:0/2949166252 wait complete. 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.593+0000 7fb83884eb80 20 client.15213 trim_cache size 0 max 16384 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 1, 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24499, 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:42.598 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 175.566778411, 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 264.08752965899998 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:42.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 msgr2=0x7fb8100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 0x7fb8100b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb81400b5c0 tx=0x7fb814005960 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 msgr2=0x7fb82419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb81c007d70 tx=0x7fb81c00a340 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 shutdown_connections 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb8100b2880 0x7fb8100b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8241080a0 0x7fb82419c010 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb824102e70 0x7fb82419b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 --2- 172.21.15.50:0/546839314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb824107750 0x7fb82419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 >> 172.21.15.50:0/546839314 conn(0x7fb824079ae0 msgr2=0x7fb824106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 shutdown_connections 2024-05-17T04:35:42.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:42.594+0000 7fb829e73700 1 -- 172.21.15.50:0/546839314 wait complete. 2024-05-17T04:35:43.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-17T04:35:43.738 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:44.149 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:43 smithi050 ceph-mon[31739]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:43 smithi105 ceph-mon[34795]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:43 smithi110 ceph-mon[35497]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:45.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 -- 172.21.15.50:0/1027100426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8107750 msgr2=0x7fe9b8107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 --2- 172.21.15.50:0/1027100426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8107750 0x7fe9b8107b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009990 tx=0x7fe9a8009ca0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 -- 172.21.15.50:0/1027100426 shutdown_connections 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 --2- 172.21.15.50:0/1027100426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b81080a0 0x7fe9b810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 --2- 172.21.15.50:0/1027100426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b8102e20 0x7fe9b8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 --2- 172.21.15.50:0/1027100426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8107750 0x7fe9b8107b60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 -- 172.21.15.50:0/1027100426 >> 172.21.15.50:0/1027100426 conn(0x7fe9b8079bd0 msgr2=0x7fe9b8079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 -- 172.21.15.50:0/1027100426 shutdown_connections 2024-05-17T04:35:45.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.638+0000 7fe9c026c700 1 -- 172.21.15.50:0/1027100426 wait complete. 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 Processor -- start 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 -- start start 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b8107750 0x7fe9b819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b81080a0 0x7fe9b819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8112b60 con 0x7fe9b8102e20 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b81129e0 con 0x7fe9b81080a0 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9c026c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8112ce0 con 0x7fe9b8107750 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9be008700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.639+0000 7fe9be008700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51088/0 (socket says 172.21.15.50:51088) 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 -- 172.21.15.50:0/3775990645 learned_addr learned my addr 172.21.15.50:0/3775990645 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:45.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be809700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b81080a0 0x7fe9b819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9bd807700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b8107750 0x7fe9b819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 -- 172.21.15.50:0/3775990645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b8107750 msgr2=0x7fe9b819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b8107750 0x7fe9b819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 -- 172.21.15.50:0/3775990645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b81080a0 msgr2=0x7fe9b819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b81080a0 0x7fe9b819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 -- 172.21.15.50:0/3775990645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b819f5a0 con 0x7fe9b8102e20 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9be008700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400d950 tx=0x7fe9b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9af7fe700 1 -- 172.21.15.50:0/3775990645 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4016070 con 0x7fe9b8102e20 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9af7fe700 1 -- 172.21.15.50:0/3775990645 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9b400fd20 con 0x7fe9b8102e20 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b819f890 con 0x7fe9b8102e20 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9af7fe700 1 -- 172.21.15.50:0/3775990645 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400e3b0 con 0x7fe9b8102e20 2024-05-17T04:35:45.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.640+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b81a71b0 con 0x7fe9b8102e20 2024-05-17T04:35:45.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.644+0000 7fe9af7fe700 1 -- 172.21.15.50:0/3775990645 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe9b4019070 con 0x7fe9b8102e20 2024-05-17T04:35:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9af7fe700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 0x7fe9a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 Processor -- start 2024-05-17T04:35:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9bd807700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 0x7fe9a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9af7fe700 1 -- 172.21.15.50:0/3775990645 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9b40e20c0 con 0x7fe9b8102e20 2024-05-17T04:35:45.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 -- start start 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e8887e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e8882570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9bd807700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 0x7fe9a40b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b5c0 tx=0x7fe9a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5589e87b1080 con 0x5589e8882ab0 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589e87a0c20 con 0x5589e88883b0 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.645+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589e87a4c60 con 0x5589e87049a0 2024-05-17T04:35:45.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51100/0 (socket says 172.21.15.50:51100) 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 -- 172.21.15.50:0/2810512053 learned_addr learned my addr 172.21.15.50:0/2810512053 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9bd807700 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e8882570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be008700 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e8887e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 -- 172.21.15.50:0/2810512053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 msgr2=0x5589e8887e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e8887e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 -- 172.21.15.50:0/2810512053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 msgr2=0x5589e8882570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e8882570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 -- 172.21.15.50:0/2810512053 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589e873a570 con 0x5589e8882ab0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9be809700 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003f60 tx=0x7fe9b0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8731846194762 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9acd46700 1 -- 172.21.15.50:0/2810512053 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004750 con 0x5589e8882ab0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9acd46700 1 -- 172.21.15.50:0/2810512053 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9b001a070 con 0x5589e8882ab0 2024-05-17T04:35:45.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.646+0000 7fe9acd46700 1 -- 172.21.15.50:0/2810512053 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00156e0 con 0x5589e8882ab0 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2810512053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 msgr2=0x5589e8882ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003f60 tx=0x7fe9b0003f90 comp rx=0 tx=0).stop 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2810512053 shutdown_connections 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8882ec0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e8887e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2810512053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e8882570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2810512053 >> 172.21.15.50:0/2810512053 conn(0x5589e8764780 msgr2=0x5589e887d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2810512053 shutdown_connections 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.647+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2810512053 wait complete. 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 -- start start 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 Processor -- start 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e888bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 0x5589e888c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5589e86a8100 con 0x5589e88883b0 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589e87428f0 con 0x5589e8882ab0 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589e888ed80 con 0x5589e87049a0 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9cec47b80 10 client.0 _ll_register_callbacks cb 0x5589e8737b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9be809700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9be809700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51116/0 (socket says 172.21.15.50:51116) 2024-05-17T04:35:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.648+0000 7fe9be809700 1 -- 172.21.15.50:0/3831188311 learned_addr learned my addr 172.21.15.50:0/3831188311 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be008700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e888bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9bd807700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 0x5589e888c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 -- 172.21.15.50:0/3831188311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 msgr2=0x5589e888bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e888bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 -- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 msgr2=0x5589e888c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 0x5589e888c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 -- 172.21.15.50:0/3831188311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589e86f1430 con 0x5589e88883b0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9bd807700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 0x5589e888c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9be809700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003e80 tx=0x7fe9b00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95649c282b0041fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0022070 con 0x5589e88883b0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9b0005050 con 0x5589e88883b0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b001a070 con 0x5589e88883b0 2024-05-17T04:35:45.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9b0005250 con 0x5589e88883b0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9cec47b80 10 client.15225 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.649+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589e8899990 con 0x5589e88883b0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9b001a070 con 0x5589e88883b0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe9b0015680 con 0x5589e88883b0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 20 client.15225 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e889bde0 0x5589e889d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 4 client.15225 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589e87b2fd0 con 0x5589e889bde0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.650+0000 7fe9be008700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e889bde0 0x5589e889d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.651+0000 7fe9be008700 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e889bde0 0x5589e889d250 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c868ec199fd2af server_cookie=2d57c342844a2a88 in_seq=0 out_seq=0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.651+0000 7fe9a1ffb700 10 client.15225 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/3831188311 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589e87b2fd0 con 0x5589e889bde0 2024-05-17T04:35:45.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9a1ffb700 10 client.15225 handle_command_reply: tid=0 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 client.15225 shutdown 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 msgr2=0x5589e88980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003e80 tx=0x7fe9b00077b0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 shutdown_connections 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e88883b0 0x5589e88980b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e889bde0 0x5589e889d250 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e888bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/3831188311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e8882ab0 0x5589e888c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 >> 172.21.15.50:0/3831188311 conn(0x5589e8764780 msgr2=0x5589e887d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9a1ffb700 0 client.15225 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 shutdown_connections 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 1 -- 172.21.15.50:0/3831188311 wait complete. 2024-05-17T04:35:45.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.652+0000 7fe9cec47b80 20 client.15225 trim_cache size 0 max 16384 2024-05-17T04:35:45.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.673+0000 7fe9cec47b80 1 Processor -- start 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.673+0000 7fe9cec47b80 1 -- start start 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e87049a0 0x5589e889ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e88883b0 0x5589e88a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5589e889b590 con 0x5589e8882ab0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589e889bc60 con 0x5589e87049a0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589e889c330 con 0x5589e88883b0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51118/0 (socket says 172.21.15.50:51118) 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 -- 172.21.15.50:0/2363475144 learned_addr learned my addr 172.21.15.50:0/2363475144 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9be008700 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e87049a0 0x5589e889ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9be809700 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e88883b0 0x5589e88a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 -- 172.21.15.50:0/2363475144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e88883b0 msgr2=0x5589e88a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e88883b0 0x5589e88a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 -- 172.21.15.50:0/2363475144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e87049a0 msgr2=0x5589e889ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e87049a0 0x5589e889ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 -- 172.21.15.50:0/2363475144 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589e8879bd0 con 0x5589e8882ab0 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.674+0000 7fe9bd807700 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806b100 tx=0x7fe9a8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a01c2b8eb2750ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9a3fff700 1 -- 172.21.15.50:0/2363475144 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a802a070 con 0x5589e8882ab0 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9a3fff700 1 -- 172.21.15.50:0/2363475144 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9a8003c50 con 0x5589e8882ab0 2024-05-17T04:35:45.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9a3fff700 1 -- 172.21.15.50:0/2363475144 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800faa0 con 0x5589e8882ab0 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2363475144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 msgr2=0x5589e8898020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9b806b100 tx=0x7fe9a8021000 comp rx=0 tx=0).stop 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2363475144 shutdown_connections 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e88883b0 0x5589e88a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e87049a0 0x5589e889ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/2363475144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e8898020 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2363475144 >> 172.21.15.50:0/2363475144 conn(0x5589e8764780 msgr2=0x5589e887ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2363475144 shutdown_connections 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.675+0000 7fe9cec47b80 1 -- 172.21.15.50:0/2363475144 wait complete. 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 -- start start 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 Processor -- start 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e88a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e88a2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5589e889d0d0 con 0x5589e8882ab0 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589e889ca00 con 0x5589e88883b0 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5589e889d7a0 con 0x5589e87049a0 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9cec47b80 10 client.0 _ll_register_callbacks cb 0x5589e8737b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9bd807700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9bd807700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51132/0 (socket says 172.21.15.50:51132) 2024-05-17T04:35:45.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9bd807700 1 -- 172.21.15.50:0/795458035 learned_addr learned my addr 172.21.15.50:0/795458035 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.676+0000 7fe9be008700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e88a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9be809700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e88a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 -- 172.21.15.50:0/795458035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 msgr2=0x5589e88a6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e88a6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 -- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 msgr2=0x5589e88a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e88a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 -- 172.21.15.50:0/795458035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589e88a2b50 con 0x5589e8882ab0 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9bd807700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8006010 tx=0x7fe9a8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713cdf09e1bc534a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:45.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a802a070 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9a800f6e0 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a8003c50 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9a800f8e0 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9cec47b80 10 client.15231 fetch_fsmap learned FSMap version 26 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589e8907120 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.677+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9a8003c50 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe9a8022070 con 0x5589e8882ab0 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 26 2024-05-17T04:35:45.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 20 client.15231 populate_metadata read hostname 'smithi050' 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e8909090 0x5589e890a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 4 client.15231 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "status"}) -- 0x5589e87ac7b0 con 0x5589e8909090 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9be008700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e8909090 0x5589e890a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9be008700 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e8909090 0x5589e890a500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70c535dd83c4e30 server_cookie=2e19415d3fc02e77 in_seq=0 out_seq=0 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.678+0000 7fe9a1ffb700 10 client.15231 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9a1ffb700 1 -- 172.21.15.50:0/795458035 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5589e87ac7b0 con 0x5589e8909090 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9a1ffb700 10 client.15231 handle_command_reply: tid=0 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 client.15231 shutdown 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 msgr2=0x5589e88a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8006010 tx=0x7fe9a8003f40 comp rx=0 tx=0).stop 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 shutdown_connections 2024-05-17T04:35:45.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589e88883b0 0x5589e88a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5589e8909090 0x5589e890a500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5589e87049a0 0x5589e88a6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 --2- 172.21.15.50:0/795458035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5589e8882ab0 0x5589e88a1ed0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9a1ffb700 0 client.15231 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 >> 172.21.15.50:0/795458035 conn(0x5589e8764780 msgr2=0x5589e887de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 shutdown_connections 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 1 -- 172.21.15.50:0/795458035 wait complete. 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.679+0000 7fe9cec47b80 20 client.15231 trim_cache size 0 max 16384 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:45.685 INFO:teuthology.orchestra.run.smithi050.stdout: "cluster_fsid": "0b3ae782-1405-11ef-bc9a-c7b262605968", 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "whoami": 0, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 24506, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "want_state": "up:active", 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "state": "up:active", 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs", 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "rank_uptime": 239.75512709, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "mdsmap_epoch": 26, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch": 89, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_epoch_barrier": 87, 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout: "uptime": 257.00867036199998 2024-05-17T04:35:45.686 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 msgr2=0x7fe9a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 0x7fe9a40b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b5c0 tx=0x7fe9a8005960 comp rx=0 tx=0).stop 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 msgr2=0x7fe9b819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400d950 tx=0x7fe9b400dc60 comp rx=0 tx=0).stop 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 shutdown_connections 2024-05-17T04:35:45.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9b81080a0 0x7fe9b819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9a40b2880 0x7fe9a40b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9b8102e20 0x7fe9b819a6d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 --2- 172.21.15.50:0/3775990645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9b8107750 0x7fe9b819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 >> 172.21.15.50:0/3775990645 conn(0x7fe9b8079bd0 msgr2=0x7fe9b806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 shutdown_connections 2024-05-17T04:35:45.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:45.680+0000 7fe9c026c700 1 -- 172.21.15.50:0/3775990645 wait complete. 2024-05-17T04:35:46.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:45 smithi050 ceph-mon[31739]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:45 smithi105 ceph-mon[34795]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:45 smithi110 ceph-mon[35497]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:46.607 INFO:teuthology.run_tasks:Running task sequential... 2024-05-17T04:35:46.616 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-17T04:35:46.616 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-17T04:35:46.617 INFO:teuthology.task.print:Enabling mgr modules 2024-05-17T04:35:46.617 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-17T04:35:46.654 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-17T04:35:46.654 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-17T04:35:46.654 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-17T04:35:46.654 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T04:35:46.654 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-17T04:35:46.655 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-17T04:35:46.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:46.655 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-17T04:35:46.673 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-17T04:35:46.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:46.674 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-17T04:35:46.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:46.742 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-05-17T04:35:46.839 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-17T04:35:46.839 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-17T04:35:47.100 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:35:48.260 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:47 smithi050 ceph-mon[31739]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:47 smithi105 ceph-mon[34795]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:48.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:47 smithi110 ceph-mon[35497]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 -- 172.21.15.50:0/1256477562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 msgr2=0x7f123410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 --2- 172.21.15.50:0/1256477562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123410fa00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b0a0 tx=0x7f122c00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 -- 172.21.15.50:0/1256477562 shutdown_connections 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 --2- 172.21.15.50:0/1256477562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123410fa00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 --2- 172.21.15.50:0/1256477562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 0x7f1234103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 --2- 172.21.15.50:0/1256477562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 0x7f1234107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 -- 172.21.15.50:0/1256477562 >> 172.21.15.50:0/1256477562 conn(0x7f1234079bd0 msgr2=0x7f1234079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:48.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 -- 172.21.15.50:0/1256477562 shutdown_connections 2024-05-17T04:35:48.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.764+0000 7f123a5bd700 1 -- 172.21.15.50:0/1256477562 wait complete. 2024-05-17T04:35:48.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 Processor -- start 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 -- start start 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 0x7f123419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 0x7f123419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112b90 con 0x7f12341080a0 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112a10 con 0x7f1234107750 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f123a5bd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1234112d10 con 0x7f1234102e20 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f1238b5a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f1238b5a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41888/0 (socket says 172.21.15.50:41888) 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f1238b5a700 1 -- 172.21.15.50:0/2242725278 learned_addr learned my addr 172.21.15.50:0/2242725278 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:48.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f122bfff700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 0x7f123419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.765+0000 7f1233fff700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 0x7f123419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 -- 172.21.15.50:0/2242725278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 msgr2=0x7f123419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 0x7f123419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 -- 172.21.15.50:0/2242725278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 msgr2=0x7f123419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 0x7f123419b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 -- 172.21.15.50:0/2242725278 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123419f520 con 0x7f12341080a0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1238b5a700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b780 tx=0x7f122c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c021070 con 0x7f12341080a0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123419f7b0 con 0x7f12341080a0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f122c007b80 con 0x7f12341080a0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c005040 con 0x7f12341080a0 2024-05-17T04:35:48.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.766+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a7130 con 0x7f12341080a0 2024-05-17T04:35:48.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.767+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12341042d0 con 0x7f12341080a0 2024-05-17T04:35:48.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.769+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f122c024070 con 0x7f12341080a0 2024-05-17T04:35:48.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.770+0000 7f1231ffb700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 0x7f12200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:48.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.770+0000 7f1233fff700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 0x7f12200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:48.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.770+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f122c00f020 con 0x7f12341080a0 2024-05-17T04:35:48.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.770+0000 7f1233fff700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 0x7f12200b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1224009c60 tx=0x7f1224009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:48.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:48.773+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f122c02b020 con 0x7f12341080a0 2024-05-17T04:35:49.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.040+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f123419b9b0 con 0x7f12341080a0 2024-05-17T04:35:49.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.040+0000 7f1231ffb700 1 -- 172.21.15.50:0/2242725278 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7f122c0ab8b0 con 0x7f12341080a0 2024-05-17T04:35:49.040 INFO:teuthology.orchestra.run.smithi050.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-17T04:35:49.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.042+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 msgr2=0x7f12200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.042+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 0x7f12200b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1224009c60 tx=0x7f1224009260 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.042+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 msgr2=0x7f123419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.042+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b780 tx=0x7f122c004d00 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 shutdown_connections 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f12200b2880 0x7f12200b4d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12341080a0 0x7f123419ee80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1234102e20 0x7f123419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 --2- 172.21.15.50:0/2242725278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1234107750 0x7f123419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 >> 172.21.15.50:0/2242725278 conn(0x7f1234079bd0 msgr2=0x7f12340ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 shutdown_connections 2024-05-17T04:35:49.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49.043+0000 7f123a5bd700 1 -- 172.21.15.50:0/2242725278 wait complete. 2024-05-17T04:35:49.727 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi050.front.sepia.ceph.com 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T04:35:49.728 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-17T04:35:49.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:49.728 DEBUG:teuthology.orchestra.run.smithi050:> ip addr 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 ::1/128 scope host 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f8 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f0 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:35:49.748 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:6b:fd:f9 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp7s0f1 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:84 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f0 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: inet 172.21.15.50/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft 41958sec preferred_lft 41958sec 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: inet6 fe80::ec4:7aff:febd:1584/64 scope link noprefixroute 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: valid_lft forever preferred_lft forever 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: link/ether 0c:c4:7a:bd:15:85 brd ff:ff:ff:ff:ff:ff 2024-05-17T04:35:49.749 INFO:teuthology.orchestra.run.smithi050.stdout: altname enp1s0f1 2024-05-17T04:35:49.749 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add name ceph-brx type bridge 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr flush dev ceph-brx 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set ceph-brx up 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T04:35:49.750 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:35:49.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:35:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:49.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:49.900 DEBUG:teuthology.orchestra.run.smithi050:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T04:35:49.927 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-05-17T04:35:49.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:49.929 DEBUG:teuthology.orchestra.run.smithi050:> route 2024-05-17T04:35:49.986 INFO:teuthology.orchestra.run.smithi050.stdout:Kernel IP routing table 2024-05-17T04:35:49.986 INFO:teuthology.orchestra.run.smithi050.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T04:35:49.986 INFO:teuthology.orchestra.run.smithi050.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-17T04:35:49.987 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-17T04:35:49.987 INFO:teuthology.orchestra.run.smithi050.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-17T04:35:49.987 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:35:50.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:49 smithi050 ceph-mon[31739]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:50.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2242725278' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:35:50.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:35:50.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:50.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:50.118 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-17T04:35:50.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:50.134 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list-id 2024-05-17T04:35:50.190 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:35:50.191 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:35:50.191 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-17T04:35:50.191 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-17T04:35:50.191 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:35:50.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:35:50.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:50.311 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-17T04:35:50.311 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:35:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:49 smithi105 ceph-mon[34795]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2242725278' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:35:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:49 smithi110 ceph-mon[35497]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:35:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2242725278' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-17T04:35:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:35:50.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:50.431 DEBUG:teuthology.orchestra.run.smithi050:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T04:35:50.431 DEBUG:teuthology.orchestra.run.smithi050:> set -e 2024-05-17T04:35:50.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set brx.0 up 2024-05-17T04:35:50.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T04:35:50.431 DEBUG:teuthology.orchestra.run.smithi050:> ') 2024-05-17T04:35:50.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T04:35:50.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:50.510 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-17T04:35:50.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T04:35:50.511 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -v /tmp/cephfs 2024-05-17T04:35:50.529 INFO:teuthology.orchestra.run.smithi050.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-17T04:35:50.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T04:35:50.529 DEBUG:teuthology.orchestra.run.smithi050:> chmod 0000 /tmp/cephfs 2024-05-17T04:35:50.584 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:35:50.652 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:35:50.708 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:35:50.709 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:35:50.710 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:50.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:50.712 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:35:50.764 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T04:35:50.764 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/ddab5ecc-ad15-4ecc-94b9-04078a785401 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-17T04:35:50.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:35:50.832 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:35:50.891 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:35:50.892 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:50.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:50.894 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:35:50.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:2024-05-17T04:35:50.904+0000 7f0304398480 -1 init, newargv = 0x7f02f40047e0 newargc=17 2024-05-17T04:35:50.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[87745]: starting ceph client 2024-05-17T04:35:50.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[87745]: starting fuse 2024-05-17T04:35:52.022 DEBUG:teuthology.orchestra.run.smithi050:> sudo modprobe fuse 2024-05-17T04:35:52.056 DEBUG:teuthology.orchestra.run.smithi050:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T04:35:52.115 INFO:teuthology.orchestra.run.smithi050.stdout:/proc 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/dev 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/security 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/shm 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/pts 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/run 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/cgroup 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/pstore 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/bpf 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/ 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/selinux 2024-05-17T04:35:52.116 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/hugepages 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/mqueue 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/debug 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/tracing 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/fs/fuse/connections 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/sys/kernel/config 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-sysctl.service 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/nfs/rpc_pipefs 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/1000 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/ceph 2024-05-17T04:35:52.117 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ed7b2d29d6d507f7edfb9e15162dcb3a4e00ba428fb2043f5a7978682a182199/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7a6f33978ea582b637a523c6ab2857923e0915caa4a4e8030dcbef9d474bb64e/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/run/user/0 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/69bd7334fe074de80a6740fb07224784eb3b8e1cc952ab245b4d95c735ef991d/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/feb1e4b87ea84e551e2c62e95e0e5bff57b9f0d39d2ff3e99a9264b46fed4071/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/7ab83483ee2d16c7f0a5384b4519af27c46eb1f26c1c21e0d2d583bb94b9b1f1/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/b4e9a258e5fcd7da84e522c6a1dd12878dfc0c02afaf3824fb939b5c6682789b/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/f8dc4a3ec5f7c4b021cdca0e1fb400d8dd061d6e240a31f79ec94cc36aff72bb/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/3c123746040a8958aa884f85eff5d5288dd71f8dcf8a269a8b89b7743a8bc476/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/4cd710214caa12d48386a73d464aa4b37c52906dc2b09963657146a7fd44fa36/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/var/lib/containers/storage/overlay/ade1bbcb60d44cb84480fd84137ba985bb2b977908e771d5d9f94ee585456528/merged 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns 2024-05-17T04:35:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:52.119 INFO:teuthology.orchestra.run.smithi050.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-17T04:35:52.119 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs 2024-05-17T04:35:52.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:52.120 DEBUG:teuthology.orchestra.run.smithi050:> ls /sys/fs/fuse/connections 2024-05-17T04:35:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:107 2024-05-17T04:35:52.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> sudo stdin-killer -- python3 -c ' 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> import glob 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> import re 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> import os 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> import subprocess 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> def _find_admin_socket(client_name): 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> files = glob.glob(asok_path) 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> mountpoint = "/tmp/cephfs" 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> # Given a non-glob path, it better be there 2024-05-17T04:35:52.179 DEBUG:teuthology.orchestra.run.smithi050:> if "*" not in asok_path: 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> assert(len(files) == 1) 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> return files[0] 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> for f in files: 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> contents = proc_f.read() 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> if mountpoint in contents: 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> return f 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> print(_find_admin_socket("client.0")) 2024-05-17T04:35:52.180 DEBUG:teuthology.orchestra.run.smithi050:> ' 2024-05-17T04:35:52.244 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:51 smithi050 ceph-mon[31739]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:52.309 INFO:teuthology.orchestra.run.smithi050.stdout:/var/run/ceph/ceph-client.0.87745.asok 2024-05-17T04:35:52.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T04:35:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:51 smithi105 ceph-mon[34795]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:52.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:51 smithi110 ceph-mon[35497]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-17T04:35:52.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87745.asok 2024-05-17T04:35:52.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:52.320 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87745.asok status 2024-05-17T04:35:52.470 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:35:52.470 INFO:teuthology.orchestra.run.smithi050.stdout: "metadata": { 2024-05-17T04:35:52.470 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "entity_id": "0", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": "smithi050", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mount_point": "/tmp/cephfs", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "pid": "87745", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "root": "/volumes/qa/sv_0/ddab5ecc-ad15-4ecc-94b9-04078a785401" 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_count": 0, 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "dentry_pinned_count": 0, 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "id": 15243, 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "inst": { 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": { 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "client", 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: "num": 15243 2024-05-17T04:35:52.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3755009313 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": { 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "192.168.0.1:0", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 3755009313 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "inst_str": "client.15243 192.168.0.1:0/3755009313", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "addr_str": "192.168.0.1:0/3755009313", 2024-05-17T04:35:52.472 INFO:teuthology.orchestra.run.smithi050.stdout: "inode_count": 5, 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_epoch": 26, 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch": 89, 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_epoch_barrier": 0, 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout: "blocklisted": false, 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout: "fs_name": "cephfs" 2024-05-17T04:35:52.473 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:35:52.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:52.479 DEBUG:teuthology.orchestra.run.smithi050:> stat --file-system '--printf=%T 2024-05-17T04:35:52.479 DEBUG:teuthology.orchestra.run.smithi050:> ' -- /tmp/cephfs 2024-05-17T04:35:52.495 INFO:teuthology.orchestra.run.smithi050.stdout:fuseblk 2024-05-17T04:35:52.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-17T04:35:52.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:35:52.496 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 1777 /tmp/cephfs 2024-05-17T04:35:52.563 INFO:teuthology.run_tasks:Running task exec... 2024-05-17T04:35:52.573 INFO:teuthology.task.exec:Executing custom commands... 2024-05-17T04:35:52.573 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:35:52.573 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 -- 172.21.15.50:0/3897217300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc01020b0 msgr2=0x7f7bc01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 --2- 172.21.15.50:0/3897217300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc01020b0 0x7f7bc01024c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009900 tx=0x7f7bb002a410 comp rx=0 tx=0).stop 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 -- 172.21.15.50:0/3897217300 shutdown_connections 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 --2- 172.21.15.50:0/3897217300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc0103b20 0x7f7bc010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 --2- 172.21.15.50:0/3897217300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01031d0 0x7f7bc01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 --2- 172.21.15.50:0/3897217300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc01020b0 0x7f7bc01024c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 -- 172.21.15.50:0/3897217300 >> 172.21.15.50:0/3897217300 conn(0x7f7bc00fd8b0 msgr2=0x7f7bc00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 -- 172.21.15.50:0/3897217300 shutdown_connections 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.719+0000 7f7bc7066640 1 -- 172.21.15.50:0/3897217300 wait complete. 2024-05-17T04:35:52.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 Processor -- start 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 -- start start 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01020b0 0x7f7bc0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc01031d0 0x7f7bc0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010da90 con 0x7f7bc0103b20 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010d910 con 0x7f7bc01031d0 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc7066640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc010dc10 con 0x7f7bc01020b0 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41904/0 (socket says 172.21.15.50:41904) 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 -- 172.21.15.50:0/781128930 learned_addr learned my addr 172.21.15.50:0/781128930 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc4e02640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01020b0 0x7f7bc0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bbffff640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc01031d0 0x7f7bc0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:52.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 -- 172.21.15.50:0/781128930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01020b0 msgr2=0x7f7bc0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01020b0 0x7f7bc0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 -- 172.21.15.50:0/781128930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc01031d0 msgr2=0x7f7bc0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc01031d0 0x7f7bc0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 -- 172.21.15.50:0/781128930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc01a1360 con 0x7f7bc0103b20 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.720+0000 7f7bc5603640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800eaa0 tx=0x7f7bb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.721+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cdb0 con 0x7f7bc0103b20 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.721+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7bb8005020 con 0x7f7bc0103b20 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.721+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc01a1650 con 0x7f7bc0103b20 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.721+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80116d0 con 0x7f7bc0103b20 2024-05-17T04:35:52.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.721+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc01a1b50 con 0x7f7bc0103b20 2024-05-17T04:35:52.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.722+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc01a1e10 con 0x7f7bc0103b20 2024-05-17T04:35:52.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.723+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7bb80051d0 con 0x7f7bc0103b20 2024-05-17T04:35:52.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.724+0000 7f7bbdffb640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 0x7f7ba80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:35:52.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.724+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7bb80e26e0 con 0x7f7bc0103b20 2024-05-17T04:35:52.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.724+0000 7f7bc4e02640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 0x7f7ba80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:35:52.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.725+0000 7f7bc4e02640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 0x7f7ba80b4cf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005ab0 tx=0x7f7bb0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:35:52.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.729+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bb8016030 con 0x7f7bc0103b20 2024-05-17T04:35:52.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:52.961+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f7bc01a1fe0 con 0x7f7bc0103b20 2024-05-17T04:35:53.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.862+0000 7f7bbdffb640 1 -- 172.21.15.50:0/781128930 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f7bb80a9a20 con 0x7f7bc0103b20 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 msgr2=0x7f7ba80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 0x7f7ba80b4cf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005ab0 tx=0x7f7bb0000bc0 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 msgr2=0x7f7bc01a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800eaa0 tx=0x7f7bb800ef60 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 shutdown_connections 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f7ba80b27e0 0x7f7ba80b4cf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7bc0103b20 0x7f7bc01a0cf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc01031d0 0x7f7bc0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.865+0000 7f7bc7066640 1 --2- 172.21.15.50:0/781128930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc01020b0 0x7f7bc0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.866+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 >> 172.21.15.50:0/781128930 conn(0x7f7bc00fd8b0 msgr2=0x7f7bc006b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:35:53.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.866+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 shutdown_connections 2024-05-17T04:35:53.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:35:53.866+0000 7f7bc7066640 1 -- 172.21.15.50:0/781128930 wait complete. 2024-05-17T04:35:53.880 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-17T04:35:53.890 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-17T04:35:53.926 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:35:53.926 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-17T04:35:53.928 INFO:teuthology.run_tasks:Running task exec... 2024-05-17T04:35:53.960 INFO:teuthology.task.exec:Executing custom commands... 2024-05-17T04:35:53.960 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:35:53.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-17T04:35:54.000 INFO:teuthology.orchestra.run.smithi050.stdout:total 0 2024-05-17T04:35:54.012 INFO:teuthology.orchestra.run.smithi050.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-17T04:35:54.012 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-fuse 1.1T 372M 1.1T 1% /tmp/cephfs 2024-05-17T04:35:54.015 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-17T04:35:54.109 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:53 smithi050 ceph-mon[31739]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T04:35:54.109 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:53 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T04:35:54.111 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-17T04:35:54.152 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-17T04:35:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:53 smithi105 ceph-mon[34795]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T04:35:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:53 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T04:35:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:53 smithi110 ceph-mon[35497]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-17T04:35:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:53 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-17T04:35:55.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:54 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T04:35:55.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:54 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:35:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:54 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T04:35:55.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:54 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:35:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:54 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/781128930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-17T04:35:55.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:54 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:35:55.693 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-17T04:35:55.732 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-17T04:35:55.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-17T04:35:55.814 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T04:35:55.900 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/: 2024-05-17T04:35:55.900 INFO:teuthology.orchestra.run.smithi050.stdout:total 2 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 .. 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 data 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data: 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:35:55.901 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 .. 2024-05-17T04:35:55.903 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-17T04:35:55.948 INFO:teuthology.orchestra.run.smithi050.stdout:[Service] 2024-05-17T04:35:55.948 INFO:teuthology.orchestra.run.smithi050.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-17T04:35:55.948 INFO:teuthology.orchestra.run.smithi050.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-17T04:35:55.951 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-17T04:35:56.039 INFO:teuthology.orchestra.run.smithi050.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-17T04:35:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:55 smithi105 ceph-mon[34795]: pgmap v427: 129 pgs: 129 active+clean; 655 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T04:35:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:55 smithi110 ceph-mon[35497]: pgmap v427: 129 pgs: 129 active+clean; 655 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T04:35:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:55 smithi050 ceph-mon[31739]: pgmap v427: 129 pgs: 129 active+clean; 655 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-05-17T04:35:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:57 smithi105 ceph-mon[34795]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-05-17T04:35:58.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:57 smithi110 ceph-mon[35497]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-05-17T04:35:58.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:57 smithi050 ceph-mon[31739]: pgmap v428: 129 pgs: 129 active+clean; 663 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-05-17T04:36:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:35:59 smithi105 ceph-mon[34795]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 37 KiB/s wr, 7 op/s 2024-05-17T04:36:00.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:35:59 smithi110 ceph-mon[35497]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 37 KiB/s wr, 7 op/s 2024-05-17T04:36:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:35:59 smithi050 ceph-mon[31739]: pgmap v429: 129 pgs: 129 active+clean; 1.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 37 KiB/s wr, 7 op/s 2024-05-17T04:36:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:01 smithi105 ceph-mon[34795]: pgmap v430: 129 pgs: 129 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-05-17T04:36:02.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:01 smithi110 ceph-mon[35497]: pgmap v430: 129 pgs: 129 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-05-17T04:36:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:01 smithi050 ceph-mon[31739]: pgmap v430: 129 pgs: 129 active+clean; 15 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-05-17T04:36:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:03 smithi105 ceph-mon[34795]: pgmap v431: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.7 MiB/s wr, 50 op/s 2024-05-17T04:36:04.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:03 smithi110 ceph-mon[35497]: pgmap v431: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.7 MiB/s wr, 50 op/s 2024-05-17T04:36:04.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:03 smithi050 ceph-mon[31739]: pgmap v431: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.7 MiB/s wr, 50 op/s 2024-05-17T04:36:05.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:04 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:05.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:04 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:05.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:04 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:05 smithi105 ceph-mon[34795]: pgmap v432: 129 pgs: 129 active+clean; 33 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 3.4 MiB/s wr, 80 op/s 2024-05-17T04:36:06.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:05 smithi110 ceph-mon[35497]: pgmap v432: 129 pgs: 129 active+clean; 33 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 3.4 MiB/s wr, 80 op/s 2024-05-17T04:36:06.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:05 smithi050 ceph-mon[31739]: pgmap v432: 129 pgs: 129 active+clean; 33 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 3.4 MiB/s wr, 80 op/s 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 -- 172.21.15.50:0/3902617976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 msgr2=0x7f0e3410b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/3902617976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e3410b530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009ca0 tx=0x7f0e2c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 -- 172.21.15.50:0/3902617976 shutdown_connections 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/3902617976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e3410b530 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/3902617976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341042e0 0x7f0e341046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/3902617976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341031c0 0x7f0e341035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 -- 172.21.15.50:0/3902617976 >> 172.21.15.50:0/3902617976 conn(0x7f0e340fe9a0 msgr2=0x7f0e34100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 -- 172.21.15.50:0/3902617976 shutdown_connections 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.091+0000 7f0e3a43c640 1 -- 172.21.15.50:0/3902617976 wait complete. 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 Processor -- start 2024-05-17T04:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 -- start start 2024-05-17T04:36:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341031c0 0x7f0e340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341042e0 0x7f0e34075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410ebf0 con 0x7f0e34104c30 2024-05-17T04:36:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410ea70 con 0x7f0e341031c0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.092+0000 7f0e3a43c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410ed70 con 0x7f0e341042e0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46072/0 (socket says 172.21.15.50:46072) 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 -- 172.21.15.50:0/2297642795 learned_addr learned my addr 172.21.15.50:0/2297642795 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341031c0 0x7f0e340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341042e0 0x7f0e34075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 -- 172.21.15.50:0/2297642795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341042e0 msgr2=0x7f0e34075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341042e0 0x7f0e34075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 -- 172.21.15.50:0/2297642795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341031c0 msgr2=0x7f0e340769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341031c0 0x7f0e340769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 -- 172.21.15.50:0/2297642795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e34075d00 con 0x7f0e34104c30 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e389d9640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098d0 tx=0x7f0e2c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e317fa640 1 -- 172.21.15.50:0/2297642795 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c003d90 con 0x7f0e34104c30 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e317fa640 1 -- 172.21.15.50:0/2297642795 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e2c0044d0 con 0x7f0e34104c30 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341a3840 con 0x7f0e34104c30 2024-05-17T04:36:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e317fa640 1 -- 172.21.15.50:0/2297642795 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c030d10 con 0x7f0e34104c30 2024-05-17T04:36:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.093+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a3ca0 con 0x7f0e34104c30 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e317fa640 1 -- 172.21.15.50:0/2297642795 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0e2c004040 con 0x7f0e34104c30 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 Processor -- start 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 -- start start 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119305b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193062c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561192fcdcc0 con 0x561192c8f2f0 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561192fbd860 con 0x561192cb81b0 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.096+0000 7f0e413ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561192fc18a0 con 0x561192c974d0 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46074/0 (socket says 172.21.15.50:46074) 2024-05-17T04:36:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 -- 172.21.15.50:0/2944759705 learned_addr learned my addr 172.21.15.50:0/2944759705 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119305b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193062c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e317fa640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 0x7f0e1c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 -- 172.21.15.50:0/2944759705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 msgr2=0x561193062c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193062c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 -- 172.21.15.50:0/2944759705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 msgr2=0x56119305b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119305b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 -- 172.21.15.50:0/2944759705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561192f85d50 con 0x561192c8f2f0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193062c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e389d9640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 0x7f0e1c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400bd10 tx=0x7f0e240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26929adb2b9f7f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e317fa640 1 -- 172.21.15.50:0/2297642795 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0e2c103b30 con 0x7f0e34104c30 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e1a7fc640 1 -- 172.21.15.50:0/2944759705 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005220 con 0x561192c8f2f0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.097+0000 7f0e1a7fc640 1 -- 172.21.15.50:0/2944759705 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e24004c30 con 0x561192c8f2f0 2024-05-17T04:36:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e1a7fc640 1 -- 172.21.15.50:0/2944759705 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240117b0 con 0x561192c8f2f0 2024-05-17T04:36:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e389d9640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 0x7f0e1c0b4cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0c8030 tx=0x7f0e2c0bf690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 -- 172.21.15.50:0/2944759705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 msgr2=0x56119305bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400bd10 tx=0x7f0e240107a0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 -- 172.21.15.50:0/2944759705 shutdown_connections 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193062c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119305bcf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2944759705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119305b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 -- 172.21.15.50:0/2944759705 >> 172.21.15.50:0/2944759705 conn(0x561192cac300 msgr2=0x5611930571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.098+0000 7f0e413ce740 1 -- 172.21.15.50:0/2944759705 shutdown_connections 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.099+0000 7f0e413ce740 1 -- 172.21.15.50:0/2944759705 wait complete. 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.099+0000 7f0e413ce740 1 -- start start 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.099+0000 7f0e413ce740 1 Processor -- start 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.099+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193065570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561192ef9ab0 con 0x561192c8f2f0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561192f7a4d0 con 0x561192cb81b0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561193063690 con 0x561192c974d0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 10 client.0 _ll_register_callbacks cb 0x561192fb8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46090/0 (socket says 172.21.15.50:46090) 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 -- 172.21.15.50:0/2994104911 learned_addr learned my addr 172.21.15.50:0/2994104911 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193065570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 -- 172.21.15.50:0/2994104911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 msgr2=0x561193065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193065570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 -- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 msgr2=0x56119306c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 -- 172.21.15.50:0/2994104911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561192f7b340 con 0x561192c8f2f0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e337fe640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e28014850 tx=0x7f0e2800c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ceeb095816295f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e137fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801a070 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0e28011bd0 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.100+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24004c30 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e28023050 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561192d23790 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e24004c30 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f0e2801d020 con 0x561192c8f2f0 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 10 client.15261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 20 client.15261 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c76d80 0x561193072880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 4 client.15261 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561192fcfc10 con 0x561192c76d80 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.101+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c76d80 0x561193072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.102+0000 7f0e137fe640 10 client.15261 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.102+0000 7f0e33fff640 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c76d80 0x561193072880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba6250aa2fe138 server_cookie=ac4a9a217a72701a in_seq=0 out_seq=0 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e137fe640 1 -- 172.21.15.50:0/2994104911 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561192fcfc10 con 0x561192c76d80 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e137fe640 10 client.15261 handle_command_reply: tid=0 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 client.15261 shutdown 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 msgr2=0x561193065030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e28014850 tx=0x7f0e2800c6b0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 shutdown_connections 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c974d0 0x561193065570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c76d80 0x561193072880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 --2- 172.21.15.50:0/2994104911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x561193065030 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e137fe640 0 client.15261 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 >> 172.21.15.50:0/2994104911 conn(0x561192cac300 msgr2=0x56119305a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 shutdown_connections 2024-05-17T04:36:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 1 -- 172.21.15.50:0/2994104911 wait complete. 2024-05-17T04:36:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.103+0000 7f0e413ce740 20 client.15261 trim_cache size 0 max 16384 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 Processor -- start 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 -- start start 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c8f2f0 0x561193057140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341b0a90 con 0x561192c974d0 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ae360 con 0x561192cb81b0 2024-05-17T04:36:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e413ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ac1b0 con 0x561192c8f2f0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46100/0 (socket says 172.21.15.50:46100) 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 -- 172.21.15.50:0/3019003521 learned_addr learned my addr 172.21.15.50:0/3019003521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e337fe640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e337fe640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c8f2f0 0x561193057140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 -- 172.21.15.50:0/3019003521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c8f2f0 msgr2=0x561193057140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c8f2f0 0x561193057140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 -- 172.21.15.50:0/3019003521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 msgr2=0x56119306c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.119+0000 7f0e33fff640 1 -- 172.21.15.50:0/3019003521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561192d9c620 con 0x561192c974d0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.120+0000 7f0e337fe640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.120+0000 7f0e33fff640 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e24010ca0 tx=0x7f0e2400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb18115c05e01db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.120+0000 7f0e19ffb640 1 -- 172.21.15.50:0/3019003521 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004020 con 0x561192c974d0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.120+0000 7f0e19ffb640 1 -- 172.21.15.50:0/3019003521 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e240041c0 con 0x561192c974d0 2024-05-17T04:36:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.120+0000 7f0e19ffb640 1 -- 172.21.15.50:0/3019003521 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402a110 con 0x561192c974d0 2024-05-17T04:36:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- 172.21.15.50:0/3019003521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 msgr2=0x56119306c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e24010ca0 tx=0x7f0e2400b410 comp rx=0 tx=0).stop 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- 172.21.15.50:0/3019003521 shutdown_connections 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192cb81b0 0x56119306c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c974d0 0x56119306c3f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 --2- 172.21.15.50:0/3019003521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192c8f2f0 0x561193057140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- 172.21.15.50:0/3019003521 >> 172.21.15.50:0/3019003521 conn(0x561192cac300 msgr2=0x561193056a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- 172.21.15.50:0/3019003521 shutdown_connections 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- 172.21.15.50:0/3019003521 wait complete. 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 -- start start 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.121+0000 7f0e413ce740 1 Processor -- start 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 0x561193069540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192cb81b0 0x561193069a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ac820 con 0x561192c8f2f0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341adc90 con 0x561192c974d0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341acef0 con 0x561192cb81b0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 10 client.0 _ll_register_callbacks cb 0x7f0e28005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46108/0 (socket says 172.21.15.50:46108) 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 -- 172.21.15.50:0/1941126589 learned_addr learned my addr 172.21.15.50:0/1941126589 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e33fff640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 0x561193069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 -- 172.21.15.50:0/1941126589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192cb81b0 msgr2=0x561193069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192cb81b0 0x561193069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 -- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 msgr2=0x561193069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 0x561193069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 -- 172.21.15.50:0/1941126589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119306a010 con 0x561192c8f2f0 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e33fff640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 0x561193069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:36:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e337fe640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0e280038d0 tx=0x7f0e28011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f07c2065089c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e137fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801a040 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0e280142d0 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.122+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e240041c0 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e28037050 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 10 client.15267 fetch_fsmap learned FSMap version 28 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611930d9c60 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e240041c0 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f0e28030060 con 0x561192c8f2f0 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 10 client.15267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 20 client.15267 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c4c1c0 0x5611930dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 4 client.15267 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:36:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561192fc93f0 con 0x561192c4c1c0 2024-05-17T04:36:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.123+0000 7f0e33fff640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c4c1c0 0x5611930dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.124+0000 7f0e33fff640 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c4c1c0 0x5611930dc6b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0c4cfebaf78ed8 server_cookie=adcb8ab1e4d1ecea in_seq=0 out_seq=0 2024-05-17T04:36:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.124+0000 7f0e137fe640 10 client.15267 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e137fe640 1 -- 172.21.15.50:0/1941126589 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561192fc93f0 con 0x561192c4c1c0 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e137fe640 10 client.15267 handle_command_reply: tid=0 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 client.15267 shutdown 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 msgr2=0x56119306c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0e280038d0 tx=0x7f0e28011bd0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 shutdown_connections 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561192cb81b0 0x561193069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561192c974d0 0x561193069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561192c4c1c0 0x5611930dc6b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 --2- 172.21.15.50:0/1941126589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561192c8f2f0 0x56119306c970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e137fe640 0 client.15267 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 >> 172.21.15.50:0/1941126589 conn(0x561192cac300 msgr2=0x56119305ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 shutdown_connections 2024-05-17T04:36:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 1 -- 172.21.15.50:0/1941126589 wait complete. 2024-05-17T04:36:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.512+0000 7f0e413ce740 20 client.15267 trim_cache size 0 max 16384 2024-05-17T04:36:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:36:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:36:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 msgr2=0x7f0e1c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 0x7f0e1c0b4cf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0c8030 tx=0x7f0e2c0bf690 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 msgr2=0x7f0e340755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098d0 tx=0x7f0e2c003ba0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 shutdown_connections 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0e1c0b27e0 0x7f0e1c0b4cf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e34104c30 0x7f0e340755d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e341042e0 0x7f0e34075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 --2- 172.21.15.50:0/2297642795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e341031c0 0x7f0e340769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 >> 172.21.15.50:0/2297642795 conn(0x7f0e340fe9a0 msgr2=0x7f0e34107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 shutdown_connections 2024-05-17T04:36:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:07.513+0000 7f0e3a43c640 1 -- 172.21.15.50:0/2297642795 wait complete. 2024-05-17T04:36:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:07 smithi105 ceph-mon[34795]: pgmap v433: 129 pgs: 129 active+clean; 35 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.9 MiB/s wr, 86 op/s 2024-05-17T04:36:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:07 smithi110 ceph-mon[35497]: pgmap v433: 129 pgs: 129 active+clean; 35 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.9 MiB/s wr, 86 op/s 2024-05-17T04:36:08.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:07 smithi050 ceph-mon[31739]: pgmap v433: 129 pgs: 129 active+clean; 35 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.9 MiB/s wr, 86 op/s 2024-05-17T04:36:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:09 smithi105 ceph-mon[34795]: pgmap v434: 129 pgs: 129 active+clean; 37 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.1 MiB/s wr, 125 op/s 2024-05-17T04:36:10.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:09 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:10.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:09 smithi110 ceph-mon[35497]: pgmap v434: 129 pgs: 129 active+clean; 37 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.1 MiB/s wr, 125 op/s 2024-05-17T04:36:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:09 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:09 smithi050 ceph-mon[31739]: pgmap v434: 129 pgs: 129 active+clean; 37 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 4.1 MiB/s wr, 125 op/s 2024-05-17T04:36:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:09 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:11.184 INFO:teuthology.orchestra.run.smithi050.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-17T04:36:11.187 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/: 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:total 2 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:35 . 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:35 .. 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 data 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data: 2024-05-17T04:36:11.504 INFO:teuthology.orchestra.run.smithi050.stdout:total 45 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 . 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:35 .. 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 17 04:36 base 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 17 04:35 global 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 log 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_commit_ts 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_dynshmem 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 17 04:35 pg_hba.conf 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 17 04:35 pg_ident.conf 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:36 pg_logical 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 pg_multixact 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_notify 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_replslot 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_serial 2024-05-17T04:36:11.505 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_snapshots 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_stat 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_stat_tmp 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 pg_subtrans 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_tblspc 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_twophase 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:35 PG_VERSION 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:35 pg_wal 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 pg_xact 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 17 04:35 postgresql.auto.conf 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 17 04:35 postgresql.conf 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base: 2024-05-17T04:36:11.506 INFO:teuthology.orchestra.run.smithi050.stdout:total 3 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 17 04:36 . 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 1 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 13493 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 13494 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:total 7778 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 . 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 17 04:36 .. 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 112 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 113 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 1247 2024-05-17T04:36:11.507 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1247_vm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:36 1249 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1249_fsm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1249_vm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:36 1259 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1259_vm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:36:11.508 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13346 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13348 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13351 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13353 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13356 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13358 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:36:11.509 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13361 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13363 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 1417 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 1418 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 174 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 175 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2187 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2224 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2328 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2336 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2337 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2579 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:36:11.510 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2604 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606 2024-05-17T04:36:11.511 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606_vm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:36 2608 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2608_vm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2610 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2610_vm 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2611 2024-05-17T04:36:11.512 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2613 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:36:11.513 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 04:36 2619 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2619_fsm 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2619_vm 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2620 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2650 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2651 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2652 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2653 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2654 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2655 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2656 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2657 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:36 2658 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 2659 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2660 2024-05-17T04:36:11.514 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2661 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2662 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2663 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2664 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2665 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2666 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2667 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2668 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2669 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2670 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:36 2673 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:36 2674 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2678 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2679 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2680 2024-05-17T04:36:11.515 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2683 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2686 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2687 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2688 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:35 2689 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2696 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2699 2024-05-17T04:36:11.516 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2701 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2702 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2703 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2704 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2754 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2755 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2756 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2757 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2830 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2831 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2832 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2833 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2834 2024-05-17T04:36:11.517 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2835 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2836 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2837 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2995 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2996 2024-05-17T04:36:11.518 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3118 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3119 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3256 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3257 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3258 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3350 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3351 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3379 2024-05-17T04:36:11.519 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3380 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3381 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3429 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3430 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3431 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3433 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3439 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3440 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3455 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:36:11.520 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3466 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3467 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3468 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3501 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3502 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3503 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3534 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3542 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3574 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3575 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3576 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3596 2024-05-17T04:36:11.521 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3597 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3598 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3599 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:36:11.522 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3606 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3607 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3997 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4143 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4144 2024-05-17T04:36:11.523 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4145 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4146 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4147 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4148 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4149 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4150 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4151 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4152 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4153 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4154 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4155 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4156 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4157 2024-05-17T04:36:11.524 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4158 2024-05-17T04:36:11.755 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4159 2024-05-17T04:36:11.755 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4160 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4161 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4162 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4163 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4164 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4165 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4166 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4167 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4168 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4169 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4170 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4171 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4172 2024-05-17T04:36:11.756 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4173 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4174 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 5002 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 548 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 549 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6102 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6104 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6106 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6110 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6111 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6112 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6113 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6117 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 826 2024-05-17T04:36:11.757 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 827 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 828 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:35 pg_filenode.map 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:35 PG_VERSION 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:total 7778 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 . 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 17 04:36 .. 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 112 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 113 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 1247 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1247_vm 2024-05-17T04:36:11.758 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:36 1249 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1249_fsm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1249_vm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:36 1259 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1259_vm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13346 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13348 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:36:11.759 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13351 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13353 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13356 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13358 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13361 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13363 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1417 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1418 2024-05-17T04:36:11.760 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 174 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 175 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2187 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2224 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2328 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2336 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2337 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2579 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:36:11.761 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2604 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606_vm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:36:11.762 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:36 2608 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2608_vm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2610 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2610_vm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2611 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2613 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:36:11.763 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 04:36 2619 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2619_fsm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2619_vm 2024-05-17T04:36:11.764 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2620 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2650 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2651 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2652 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2653 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2654 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2655 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2656 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2657 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:36 2658 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 2659 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2660 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2661 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2662 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2663 2024-05-17T04:36:11.765 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2664 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2665 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2666 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2667 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2668 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2669 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2670 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:36 2673 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:36 2674 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2678 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2679 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2680 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:36:11.766 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2683 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2686 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2687 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2688 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2689 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2696 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2699 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2701 2024-05-17T04:36:11.767 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2702 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2703 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2704 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2754 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2755 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2756 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2757 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2830 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2831 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2832 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2833 2024-05-17T04:36:11.768 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2834 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2835 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2836 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2837 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2995 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2996 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:36:11.769 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3118 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3119 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3256 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3257 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3258 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3350 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3351 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3379 2024-05-17T04:36:11.770 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3380 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3381 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3429 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3430 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3431 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3433 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3439 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3440 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3455 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:36:11.771 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3466 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3467 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3468 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3501 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3502 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3503 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3534 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3542 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3574 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3575 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3576 2024-05-17T04:36:11.772 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3596 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3597 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3598 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3599 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:36:11.773 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3606 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3607 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3997 2024-05-17T04:36:11.774 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4143 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4144 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4145 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4146 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4147 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4148 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4149 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4150 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4151 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4152 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4153 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4154 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4155 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4156 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4157 2024-05-17T04:36:11.775 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4158 2024-05-17T04:36:11.776 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4159 2024-05-17T04:36:11.776 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4160 2024-05-17T04:36:11.776 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4161 2024-05-17T04:36:11.776 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4162 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4163 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4164 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4165 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4166 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4167 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4168 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4169 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4170 2024-05-17T04:36:12.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4171 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4172 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4173 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4174 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 5002 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 548 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 549 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6102 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6104 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6106 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6110 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6111 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6112 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6113 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6117 2024-05-17T04:36:12.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 826 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 827 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 828 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:36 pg_filenode.map 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:36 PG_VERSION 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:total 7778 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 . 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 17 04:36 .. 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 112 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 113 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 1247 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:36:12.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1247_vm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:36 1249 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1249_fsm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1249_vm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:36 1259 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1259_vm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13346 2024-05-17T04:36:12.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13348 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13351 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13353 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13356 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13358 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13361 2024-05-17T04:36:12.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13363 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1417 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1418 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 174 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 175 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2187 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2224 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2328 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2336 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2337 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2579 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:36:12.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2604 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:36:12.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606_vm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:36 2608 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2608_vm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2610 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2610_vm 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2611 2024-05-17T04:36:12.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2613 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:36:12.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 04:36 2619 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2619_fsm 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2619_vm 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2620 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2650 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2651 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2652 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2653 2024-05-17T04:36:12.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2654 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2655 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2656 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2657 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:36 2658 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 2659 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2660 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2661 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2662 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2663 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2664 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2665 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2666 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2667 2024-05-17T04:36:12.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2668 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2669 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2670 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:36 2673 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:36 2674 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2678 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2679 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2680 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2683 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:36:12.030 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2686 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2687 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2688 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2689 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2696 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2699 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2701 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2702 2024-05-17T04:36:12.031 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2703 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2704 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2754 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2755 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2756 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2757 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2830 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2831 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2832 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2833 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2834 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2835 2024-05-17T04:36:12.032 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2836 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2837 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2995 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2996 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:36:12.033 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3118 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3119 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3256 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3257 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3258 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3350 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3351 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3379 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3380 2024-05-17T04:36:12.034 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3381 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3429 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3430 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3431 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3433 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3439 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3440 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3455 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:36:12.035 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3466 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3467 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3468 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3501 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3502 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3503 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3534 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3542 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3574 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3575 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3576 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3596 2024-05-17T04:36:12.036 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3597 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3598 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3599 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:36:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3606 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3607 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3997 2024-05-17T04:36:12.038 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4143 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4144 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4145 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4146 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4147 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4148 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4149 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4150 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4151 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4152 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4153 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4154 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4155 2024-05-17T04:36:12.039 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4156 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4157 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4158 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4159 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4160 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4161 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4162 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4163 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4164 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4165 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4166 2024-05-17T04:36:12.040 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4167 2024-05-17T04:36:12.061 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4168 2024-05-17T04:36:12.061 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4169 2024-05-17T04:36:12.061 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4170 2024-05-17T04:36:12.061 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4171 2024-05-17T04:36:12.061 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4172 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4173 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4174 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 5002 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 548 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 549 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6102 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6104 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6106 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6110 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6111 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6112 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6113 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6117 2024-05-17T04:36:12.062 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 826 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 827 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 828 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:36 pg_filenode.map 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:36 PG_VERSION 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/global: 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:total 538 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 17 04:35 . 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1213 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1213_fsm 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1213_vm 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1214 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1214_fsm 2024-05-17T04:36:12.063 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1214_vm 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 1232 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 1233 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1260 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1260_fsm 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1260_vm 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1261 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1261_fsm 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1261_vm 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1262 2024-05-17T04:36:12.064 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1262_fsm 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1262_vm 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2396 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2396_fsm 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2396_vm 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2397 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2671 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2672 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2676 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2677 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2694 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2695 2024-05-17T04:36:12.065 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2697 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2698 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2846 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2847 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2964 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2965 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2966 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2967 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3592 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3593 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4060 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4061 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4175 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4176 2024-05-17T04:36:12.066 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4177 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4178 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4181 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4182 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4183 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4184 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4185 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4186 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6000 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6001 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6002 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6100 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6114 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6115 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 pg_control 2024-05-17T04:36:12.067 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:35 pg_filenode.map 2024-05-17T04:36:12.068 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.068 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/log: 2024-05-17T04:36:12.068 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.068 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 . 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-17T04:36:12.118 INFO:teuthology.orchestra.run.smithi050.stdout:total 3 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:36 . 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 mappings 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:36 replorigin_checkpoint 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 snapshots 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:36 .. 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.119 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:36 .. 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:total 2 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 . 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 members 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 offsets 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 .. 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 0000 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.120 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 .. 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 0000 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.122 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.123 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-17T04:36:12.135 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:36:12.135 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:36:12.135 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 0000 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:total 16386 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:35 . 2024-05-17T04:36:12.136 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:36 000000010000000000000001 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 archive_status 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:35 .. 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 17 04:36 .. 2024-05-17T04:36:12.137 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 0000 2024-05-17T04:36:12.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-17T04:36:12.160 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:11 smithi050 ceph-mon[31739]: pgmap v435: 129 pgs: 129 active+clean; 44 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T04:36:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:11 smithi105 ceph-mon[34795]: pgmap v435: 129 pgs: 129 active+clean; 44 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T04:36:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:11 smithi110 ceph-mon[35497]: pgmap v435: 129 pgs: 129 active+clean; 44 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.9 MiB/s wr, 257 op/s 2024-05-17T04:36:12.322 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-17T04:36:12.474 INFO:teuthology.orchestra.run.smithi050.stderr:dropping old tables... 2024-05-17T04:36:12.487 INFO:teuthology.orchestra.run.smithi050.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-17T04:36:12.487 INFO:teuthology.orchestra.run.smithi050.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-17T04:36:12.487 INFO:teuthology.orchestra.run.smithi050.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-17T04:36:12.487 INFO:teuthology.orchestra.run.smithi050.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-17T04:36:12.487 INFO:teuthology.orchestra.run.smithi050.stderr:creating tables... 2024-05-17T04:36:12.578 INFO:teuthology.orchestra.run.smithi050.stderr:generating data (client-side)... 2024-05-17T04:36:13.182 INFO:teuthology.orchestra.run.smithi050.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.55 s, remaining 16.97 s) 2024-05-17T04:36:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:13 smithi105 ceph-mon[34795]: pgmap v436: 129 pgs: 129 active+clean; 45 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.6 MiB/s wr, 296 op/s 2024-05-17T04:36:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:13 smithi110 ceph-mon[35497]: pgmap v436: 129 pgs: 129 active+clean; 45 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.6 MiB/s wr, 296 op/s 2024-05-17T04:36:14.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:13 smithi050 ceph-mon[31739]: pgmap v436: 129 pgs: 129 active+clean; 45 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 3.6 MiB/s wr, 296 op/s 2024-05-17T04:36:14.453 INFO:teuthology.orchestra.run.smithi050.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.82 s, remaining 27.28 s) 2024-05-17T04:36:15.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:14 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:15.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:14 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:15.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:14 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:36:15.742 INFO:teuthology.orchestra.run.smithi050.stderr:300000 of 3200000 tuples (9%) done (elapsed 3.11 s, remaining 30.04 s) 2024-05-17T04:36:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:15 smithi105 ceph-mon[34795]: pgmap v437: 129 pgs: 129 active+clean; 60 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-17T04:36:16.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:15 smithi110 ceph-mon[35497]: pgmap v437: 129 pgs: 129 active+clean; 60 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-17T04:36:16.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:15 smithi050 ceph-mon[31739]: pgmap v437: 129 pgs: 129 active+clean; 60 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 4.0 MiB/s wr, 340 op/s 2024-05-17T04:36:16.508 INFO:teuthology.orchestra.run.smithi050.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.87 s, remaining 27.11 s) 2024-05-17T04:36:17.788 INFO:teuthology.orchestra.run.smithi050.stderr:500000 of 3200000 tuples (15%) done (elapsed 5.15 s, remaining 27.83 s) 2024-05-17T04:36:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:17 smithi105 ceph-mon[34795]: pgmap v438: 129 pgs: 129 active+clean; 88 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 8.2 MiB/s wr, 311 op/s 2024-05-17T04:36:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:17 smithi110 ceph-mon[35497]: pgmap v438: 129 pgs: 129 active+clean; 88 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 8.2 MiB/s wr, 311 op/s 2024-05-17T04:36:18.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:17 smithi050 ceph-mon[31739]: pgmap v438: 129 pgs: 129 active+clean; 88 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 8.2 MiB/s wr, 311 op/s 2024-05-17T04:36:18.668 INFO:teuthology.orchestra.run.smithi050.stderr:600000 of 3200000 tuples (18%) done (elapsed 6.03 s, remaining 26.15 s) 2024-05-17T04:36:19.876 INFO:teuthology.orchestra.run.smithi050.stderr:700000 of 3200000 tuples (21%) done (elapsed 7.24 s, remaining 25.86 s) 2024-05-17T04:36:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:19 smithi105 ceph-mon[34795]: pgmap v439: 129 pgs: 129 active+clean; 116 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 12 MiB/s wr, 306 op/s 2024-05-17T04:36:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:19 smithi110 ceph-mon[35497]: pgmap v439: 129 pgs: 129 active+clean; 116 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 12 MiB/s wr, 306 op/s 2024-05-17T04:36:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:19 smithi050 ceph-mon[31739]: pgmap v439: 129 pgs: 129 active+clean; 116 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 12 MiB/s wr, 306 op/s 2024-05-17T04:36:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 -- 172.21.15.50:0/616151665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 msgr2=0x7f927410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 --2- 172.21.15.50:0/616151665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f927410b350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f926c009ca0 tx=0x7f926c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 -- 172.21.15.50:0/616151665 shutdown_connections 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 --2- 172.21.15.50:0/616151665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f927410b350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 --2- 172.21.15.50:0/616151665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92741065d0 0x7f92741089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 --2- 172.21.15.50:0/616151665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9274103c70 0x7f9274106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 -- 172.21.15.50:0/616151665 >> 172.21.15.50:0/616151665 conn(0x7f92740fd880 msgr2=0x7f92740ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 -- 172.21.15.50:0/616151665 shutdown_connections 2024-05-17T04:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.629+0000 7f927a5eb640 1 -- 172.21.15.50:0/616151665 wait complete. 2024-05-17T04:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 Processor -- start 2024-05-17T04:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 -- start start 2024-05-17T04:36:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9274103c70 0x7f9274199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 0x7f9274199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927410eba0 con 0x7f9274108f30 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927410ea20 con 0x7f92741065d0 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f927a5eb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f927410ed20 con 0x7f9274103c70 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f9278b88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f9278b88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54114/0 (socket says 172.21.15.50:54114) 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f9278b88640 1 -- 172.21.15.50:0/2850943299 learned_addr learned my addr 172.21.15.50:0/2850943299 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.630+0000 7f92737fe640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 0x7f9274199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9273fff640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9274103c70 0x7f9274199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 -- 172.21.15.50:0/2850943299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9274103c70 msgr2=0x7f9274199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9274103c70 0x7f9274199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 -- 172.21.15.50:0/2850943299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 msgr2=0x7f9274199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 0x7f9274199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 -- 172.21.15.50:0/2850943299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92741a1290 con 0x7f9274108f30 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f92737fe640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 0x7f9274199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f9278b88640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f926c005e30 tx=0x7f926c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f92717fa640 1 -- 172.21.15.50:0/2850943299 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c032e90 con 0x7f9274108f30 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f92717fa640 1 -- 172.21.15.50:0/2850943299 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926c0044d0 con 0x7f9274108f30 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f92717fa640 1 -- 172.21.15.50:0/2850943299 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c030ce0 con 0x7f9274108f30 2024-05-17T04:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92741a1520 con 0x7f9274108f30 2024-05-17T04:36:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.631+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92741a1a20 con 0x7f9274108f30 2024-05-17T04:36:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.634+0000 7f928157d740 1 Processor -- start 2024-05-17T04:36:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f92717fa640 1 -- 172.21.15.50:0/2850943299 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f926c004040 con 0x7f9274108f30 2024-05-17T04:36:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 -- start start 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9603bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96043020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a95fa9cc0 con 0x561a95c6b2f0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561a95f99860 con 0x561a95c941b0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f928157d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561a95f9d8a0 con 0x561a95c734d0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54130/0 (socket says 172.21.15.50:54130) 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f92737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9603bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 -- 172.21.15.50:0/1088035542 learned_addr learned my addr 172.21.15.50:0/1088035542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f92737fe640 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96043020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 -- 172.21.15.50:0/1088035542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 msgr2=0x561a96043020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96043020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 -- 172.21.15.50:0/1088035542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 msgr2=0x561a9603bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9603bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f9273fff640 1 -- 172.21.15.50:0/1088035542 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a95f61d50 con 0x561a95c6b2f0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.635+0000 7f92737fe640 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9603bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f92717fa640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 0x7f925c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f9273fff640 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9264009ca0 tx=0x7f9264011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c7a3c7ec85141d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f92737fe640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 0x7f925c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f92717fa640 1 -- 172.21.15.50:0/2850943299 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f926c103b90 con 0x7f9274108f30 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f925a7fc640 1 -- 172.21.15.50:0/1088035542 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264004220 con 0x561a95c6b2f0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f925a7fc640 1 -- 172.21.15.50:0/1088035542 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92640043c0 con 0x561a95c6b2f0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.636+0000 7f925a7fc640 1 -- 172.21.15.50:0/1088035542 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264019670 con 0x561a95c6b2f0 2024-05-17T04:36:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- 172.21.15.50:0/1088035542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 msgr2=0x561a9603c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9264009ca0 tx=0x7f9264011810 comp rx=0 tx=0).stop 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f92737fe640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 0x7f925c0b4d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x561a9603cd70 tx=0x7f926800bb30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- 172.21.15.50:0/1088035542 shutdown_connections 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96043020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a9603c100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 --2- 172.21.15.50:0/1088035542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9603bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- 172.21.15.50:0/1088035542 >> 172.21.15.50:0/1088035542 conn(0x561a95c88300 msgr2=0x561a960331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- 172.21.15.50:0/1088035542 shutdown_connections 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- 172.21.15.50:0/1088035542 wait complete. 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.637+0000 7f928157d740 1 -- start start 2024-05-17T04:36:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 Processor -- start 2024-05-17T04:36:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96045940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a95ed5ab0 con 0x561a95c6b2f0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561a95f564d0 con 0x561a95c941b0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561a96043af0 con 0x561a95c734d0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f928157d740 10 client.0 _ll_register_callbacks cb 0x561a95f94950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54142/0 (socket says 172.21.15.50:54142) 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f9273fff640 1 -- 172.21.15.50:0/1809224424 learned_addr learned my addr 172.21.15.50:0/1809224424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f9278b88640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.638+0000 7f92737fe640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96045940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 -- 172.21.15.50:0/1809224424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 msgr2=0x561a96045940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96045940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 -- 172.21.15.50:0/1809224424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 msgr2=0x561a96045400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 -- 172.21.15.50:0/1809224424 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a95f57340 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9273fff640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f9278b88640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604c8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f926c101300 tx=0x7f926c0ca590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb76e53707ac3461 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f92537fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0cb520 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f926c0305c0 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92640043c0 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f926c0cae20 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f928157d740 10 client.25067 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.639+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a95cff790 con 0x561a95c941b0 2024-05-17T04:36:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92640043c0 con 0x561a95c941b0 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f926c0d37f0 con 0x561a95c941b0 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 10 client.25067 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 10 client.25067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 20 client.25067 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c52d80 0x561a96052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 4 client.25067 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a95fabc10 con 0x561a95c52d80 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.640+0000 7f9273fff640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c52d80 0x561a96052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.641+0000 7f92537fe640 10 client.25067 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.641+0000 7f9273fff640 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c52d80 0x561a96052cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1fa1425aac73e8 server_cookie=be26e7fa5ff70e26 in_seq=0 out_seq=0 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f92537fe640 1 -- 172.21.15.50:0/1809224424 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a95fabc10 con 0x561a95c52d80 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f92537fe640 10 client.25067 handle_command_reply: tid=0 2024-05-17T04:36:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 client.25067 shutdown 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 msgr2=0x561a9604c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604c8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f926c101300 tx=0x7f926c0ca590 comp rx=0 tx=0).stop 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 shutdown_connections 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604c8a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96045940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c52d80 0x561a96052cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 --2- 172.21.15.50:0/1809224424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96045400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f92537fe640 0 client.25067 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 >> 172.21.15.50:0/1809224424 conn(0x561a95c88300 msgr2=0x561a96036010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 shutdown_connections 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 1 -- 172.21.15.50:0/1809224424 wait complete. 2024-05-17T04:36:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.642+0000 7f928157d740 20 client.25067 trim_cache size 0 max 16384 2024-05-17T04:36:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 Processor -- start 2024-05-17T04:36:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 -- start start 2024-05-17T04:36:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 0x561a96033390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96031ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92741ac6d0 con 0x561a95c941b0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92741a9f70 con 0x561a95c6b2f0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f928157d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92741a7dc0 con 0x561a95c734d0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f9278b88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f9278b88640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54144/0 (socket says 172.21.15.50:54144) 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.665+0000 7f9278b88640 1 -- 172.21.15.50:0/1187187889 learned_addr learned my addr 172.21.15.50:0/1187187889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f92737fe640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96031ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9273fff640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 0x561a96033390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 -- 172.21.15.50:0/1187187889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 msgr2=0x561a96031ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96031ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 -- 172.21.15.50:0/1187187889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 msgr2=0x561a96033390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 0x561a96033390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 -- 172.21.15.50:0/1187187889 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a95d78620 con 0x561a95c941b0 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9273fff640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 0x561a96033390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9278b88640 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f926c0c8090 tx=0x7f926c009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0856035c6cc33de server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9259ffb640 1 -- 172.21.15.50:0/1187187889 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004330 con 0x561a95c941b0 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9259ffb640 1 -- 172.21.15.50:0/1187187889 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926c036030 con 0x561a95c941b0 2024-05-17T04:36:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.666+0000 7f9259ffb640 1 -- 172.21.15.50:0/1187187889 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0d37f0 con 0x561a95c941b0 2024-05-17T04:36:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 -- 172.21.15.50:0/1187187889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 msgr2=0x561a96032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f926c0c8090 tx=0x7f926c009460 comp rx=0 tx=0).stop 2024-05-17T04:36:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 -- 172.21.15.50:0/1187187889 shutdown_connections 2024-05-17T04:36:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c941b0 0x561a96032430 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96031ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 --2- 172.21.15.50:0/1187187889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c6b2f0 0x561a96033390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 -- 172.21.15.50:0/1187187889 >> 172.21.15.50:0/1187187889 conn(0x561a95c88300 msgr2=0x561a960317e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 -- 172.21.15.50:0/1187187889 shutdown_connections 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.667+0000 7f928157d740 1 -- 172.21.15.50:0/1187187889 wait complete. 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 -- start start 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 Processor -- start 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96049940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92741a8430 con 0x561a95c6b2f0 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92741a98a0 con 0x561a95c941b0 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92741a8b00 con 0x561a95c734d0 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f928157d740 10 client.0 _ll_register_callbacks cb 0x7f926c0048d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.668+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54150/0 (socket says 172.21.15.50:54150) 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 -- 172.21.15.50:0/2820533561 learned_addr learned my addr 172.21.15.50:0/2820533561 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f92737fe640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96049940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9278b88640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 -- 172.21.15.50:0/2820533561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 msgr2=0x561a96049940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96049940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 -- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 msgr2=0x561a9604a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 -- 172.21.15.50:0/2820533561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9604adf0 con 0x561a95c6b2f0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9278b88640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f9273fff640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f92640058c0 tx=0x7f9264009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a01c77eedd70c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f92537fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264005640 con 0x561a95c6b2f0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9264003670 con 0x561a95c6b2f0 2024-05-17T04:36:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.669+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c036030 con 0x561a95c6b2f0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9264003870 con 0x561a95c6b2f0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 10 client.15285 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a9604c120 con 0x561a95c6b2f0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926c036030 con 0x561a95c6b2f0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f926402b770 con 0x561a95c6b2f0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 10 client.15285 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 10 client.15285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 20 client.15285 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c281c0 0x561a960bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 4 client.15285 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a95fa53f0 con 0x561a95c281c0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.670+0000 7f92737fe640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c281c0 0x561a960bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.671+0000 7f92537fe640 10 client.15285 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.671+0000 7f92737fe640 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c281c0 0x561a960bd5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a196180883b4d11 server_cookie=29752f1f72fd6b6e in_seq=0 out_seq=0 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.729+0000 7f92537fe640 1 -- 172.21.15.50:0/2820533561 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a95fa53f0 con 0x561a95c281c0 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.729+0000 7f92537fe640 10 client.15285 handle_command_reply: tid=0 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.729+0000 7f928157d740 1 client.15285 shutdown 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 msgr2=0x561a96049400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f92640058c0 tx=0x7f9264009c50 comp rx=0 tx=0).stop 2024-05-17T04:36:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 shutdown_connections 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a95c941b0 0x561a9604a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561a95c734d0 0x561a96049940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561a95c281c0 0x561a960bd5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 --2- 172.21.15.50:0/2820533561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a95c6b2f0 0x561a96049400 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f92537fe640 0 client.15285 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 >> 172.21.15.50:0/2820533561 conn(0x561a95c88300 msgr2=0x561a9603edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 shutdown_connections 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 1 -- 172.21.15.50:0/2820533561 wait complete. 2024-05-17T04:36:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f928157d740 20 client.15285 trim_cache size 0 max 16384 2024-05-17T04:36:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:36:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:36:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 msgr2=0x7f925c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 0x7f925c0b4d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x561a9603cd70 tx=0x7f926800bb30 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.730+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 msgr2=0x7f92741a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f926c005e30 tx=0x7f926c003ba0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 shutdown_connections 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f925c0b2860 0x7f925c0b4d70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9274108f30 0x7f92741a0c20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741065d0 0x7f9274199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 --2- 172.21.15.50:0/2850943299 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9274103c70 0x7f9274199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 >> 172.21.15.50:0/2850943299 conn(0x7f92740fd880 msgr2=0x7f9274104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 shutdown_connections 2024-05-17T04:36:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:20.731+0000 7f927a5eb640 1 -- 172.21.15.50:0/2850943299 wait complete. 2024-05-17T04:36:21.182 INFO:teuthology.orchestra.run.smithi050.stderr:800000 of 3200000 tuples (25%) done (elapsed 8.55 s, remaining 25.64 s) 2024-05-17T04:36:22.108 INFO:teuthology.orchestra.run.smithi050.stderr:900000 of 3200000 tuples (28%) done (elapsed 9.47 s, remaining 24.21 s) 2024-05-17T04:36:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:21 smithi105 ceph-mon[34795]: pgmap v440: 129 pgs: 129 active+clean; 177 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 20 MiB/s wr, 276 op/s 2024-05-17T04:36:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:21 smithi110 ceph-mon[35497]: pgmap v440: 129 pgs: 129 active+clean; 177 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 20 MiB/s wr, 276 op/s 2024-05-17T04:36:22.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:21 smithi050 ceph-mon[31739]: pgmap v440: 129 pgs: 129 active+clean; 177 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 20 MiB/s wr, 276 op/s 2024-05-17T04:36:23.349 INFO:teuthology.orchestra.run.smithi050.stderr:1000000 of 3200000 tuples (31%) done (elapsed 10.71 s, remaining 23.57 s) 2024-05-17T04:36:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:23 smithi105 ceph-mon[34795]: pgmap v441: 129 pgs: 129 active+clean; 214 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 MiB/s wr, 155 op/s 2024-05-17T04:36:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:23 smithi110 ceph-mon[35497]: pgmap v441: 129 pgs: 129 active+clean; 214 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 MiB/s wr, 155 op/s 2024-05-17T04:36:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:23 smithi050 ceph-mon[31739]: pgmap v441: 129 pgs: 129 active+clean; 214 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 26 MiB/s wr, 155 op/s 2024-05-17T04:36:24.614 INFO:teuthology.orchestra.run.smithi050.stderr:1100000 of 3200000 tuples (34%) done (elapsed 11.98 s, remaining 22.87 s) 2024-05-17T04:36:25.467 INFO:teuthology.orchestra.run.smithi050.stderr:1200000 of 3200000 tuples (37%) done (elapsed 12.83 s, remaining 21.39 s) 2024-05-17T04:36:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:25 smithi105 ceph-mon[34795]: pgmap v442: 129 pgs: 129 active+clean; 255 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 34 MiB/s wr, 121 op/s 2024-05-17T04:36:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:25 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:36:26.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:25 smithi110 ceph-mon[35497]: pgmap v442: 129 pgs: 129 active+clean; 255 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 34 MiB/s wr, 121 op/s 2024-05-17T04:36:26.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:25 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:36:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:25 smithi050 ceph-mon[31739]: pgmap v442: 129 pgs: 129 active+clean; 255 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 34 MiB/s wr, 121 op/s 2024-05-17T04:36:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:25 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:36:26.716 INFO:teuthology.orchestra.run.smithi050.stderr:1300000 of 3200000 tuples (40%) done (elapsed 14.08 s, remaining 20.58 s) 2024-05-17T04:36:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:36:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:36:27.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:26 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:36:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:36:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:36:27.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:26 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:36:27.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:36:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:36:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:26 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:36:27.848 INFO:teuthology.orchestra.run.smithi050.stderr:1400000 of 3200000 tuples (43%) done (elapsed 15.21 s, remaining 19.56 s) 2024-05-17T04:36:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:27 smithi105 ceph-mon[34795]: pgmap v443: 129 pgs: 129 active+clean; 293 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 39 MiB/s wr, 62 op/s 2024-05-17T04:36:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:27 smithi110 ceph-mon[35497]: pgmap v443: 129 pgs: 129 active+clean; 293 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 39 MiB/s wr, 62 op/s 2024-05-17T04:36:28.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:27 smithi050 ceph-mon[31739]: pgmap v443: 129 pgs: 129 active+clean; 293 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 39 MiB/s wr, 62 op/s 2024-05-17T04:36:28.745 INFO:teuthology.orchestra.run.smithi050.stderr:1500000 of 3200000 tuples (46%) done (elapsed 16.11 s, remaining 18.26 s) 2024-05-17T04:36:30.014 INFO:teuthology.orchestra.run.smithi050.stderr:1600000 of 3200000 tuples (50%) done (elapsed 17.38 s, remaining 17.38 s) 2024-05-17T04:36:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:29 smithi105 ceph-mon[34795]: pgmap v444: 129 pgs: 129 active+clean; 329 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 62 op/s 2024-05-17T04:36:30.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:29 smithi110 ceph-mon[35497]: pgmap v444: 129 pgs: 129 active+clean; 329 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 62 op/s 2024-05-17T04:36:30.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:29 smithi050 ceph-mon[31739]: pgmap v444: 129 pgs: 129 active+clean; 329 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 40 MiB/s wr, 62 op/s 2024-05-17T04:36:30.800 INFO:teuthology.orchestra.run.smithi050.stderr:1700000 of 3200000 tuples (53%) done (elapsed 18.16 s, remaining 16.03 s) 2024-05-17T04:36:32.055 INFO:teuthology.orchestra.run.smithi050.stderr:1800000 of 3200000 tuples (56%) done (elapsed 19.42 s, remaining 15.10 s) 2024-05-17T04:36:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:31 smithi105 ceph-mon[34795]: pgmap v445: 129 pgs: 129 active+clean; 357 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 40 MiB/s wr, 68 op/s 2024-05-17T04:36:32.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:31 smithi110 ceph-mon[35497]: pgmap v445: 129 pgs: 129 active+clean; 357 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 40 MiB/s wr, 68 op/s 2024-05-17T04:36:32.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:31 smithi050 ceph-mon[31739]: pgmap v445: 129 pgs: 129 active+clean; 357 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 40 MiB/s wr, 68 op/s 2024-05-17T04:36:33.296 INFO:teuthology.orchestra.run.smithi050.stderr:1900000 of 3200000 tuples (59%) done (elapsed 20.66 s, remaining 14.14 s) 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 -- 172.21.15.50:0/2062625374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1960103190 msgr2=0x7f19601035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 --2- 172.21.15.50:0/2062625374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1960103190 0x7f19601035a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c02a410 comp rx=0 tx=0).stop 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 -- 172.21.15.50:0/2062625374 shutdown_connections 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 --2- 172.21.15.50:0/2062625374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 0x7f196010b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 --2- 172.21.15.50:0/2062625374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19601042b0 0x7f19601046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 --2- 172.21.15.50:0/2062625374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1960103190 0x7f19601035a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 -- 172.21.15.50:0/2062625374 >> 172.21.15.50:0/2062625374 conn(0x7f19600fe970 msgr2=0x7f1960100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 -- 172.21.15.50:0/2062625374 shutdown_connections 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.848+0000 7f1967e22640 1 -- 172.21.15.50:0/2062625374 wait complete. 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 Processor -- start 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 -- start start 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1960103190 0x7f19600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 0x7f19600755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f196010eb70 con 0x7f19601042b0 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e9f0 con 0x7f1960103190 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1967e22640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f196010ecf0 con 0x7f1960104c00 2024-05-17T04:36:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47090/0 (socket says 172.21.15.50:47090) 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 -- 172.21.15.50:0/2098654963 learned_addr learned my addr 172.21.15.50:0/2098654963 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f1965bbe640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1960103190 0x7f19600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19663bf640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 0x7f19600755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 -- 172.21.15.50:0/2098654963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 msgr2=0x7f19600755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 0x7f19600755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 -- 172.21.15.50:0/2098654963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1960103190 msgr2=0x7f19600769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1960103190 0x7f19600769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.849+0000 7f19653bd640 1 -- 172.21.15.50:0/2098654963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1960075d00 con 0x7f19601042b0 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f19653bd640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f195000d950 tx=0x7f195000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f1956ffd640 1 -- 172.21.15.50:0/2098654963 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950017030 con 0x7f19601042b0 2024-05-17T04:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f1956ffd640 1 -- 172.21.15.50:0/2098654963 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1950004020 con 0x7f19601042b0 2024-05-17T04:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a38b0 con 0x7f19601042b0 2024-05-17T04:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f1956ffd640 1 -- 172.21.15.50:0/2098654963 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500055d0 con 0x7f19601042b0 2024-05-17T04:36:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.850+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a3d70 con 0x7f19601042b0 2024-05-17T04:36:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.853+0000 7f196edb2740 1 Processor -- start 2024-05-17T04:36:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.853+0000 7f1956ffd640 1 -- 172.21.15.50:0/2098654963 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19500047a0 con 0x7f19601042b0 2024-05-17T04:36:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.853+0000 7f196edb2740 1 -- start start 2024-05-17T04:36:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e13546cc0 con 0x562e132082f0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e13536860 con 0x562e132104d0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f196edb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e1353a8a0 con 0x562e132311b0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47102/0 (socket says 172.21.15.50:47102) 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 -- 172.21.15.50:0/524535886 learned_addr learned my addr 172.21.15.50:0/524535886 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f19663bf640 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f19653bd640 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1956ffd640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 0x7f194c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 -- 172.21.15.50:0/524535886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 msgr2=0x562e135d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 -- 172.21.15.50:0/524535886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 msgr2=0x562e135dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1965bbe640 1 -- 172.21.15.50:0/524535886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e134fed50 con 0x562e132082f0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f19663bf640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 0x7f194c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.854+0000 7f1956ffd640 1 -- 172.21.15.50:0/2098654963 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f19500e2060 con 0x7f19601042b0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.855+0000 7f1965bbe640 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f195c031460 tx=0x7f195c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cec59fc0e6000e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.855+0000 7f194bfff640 1 -- 172.21.15.50:0/524535886 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035030 con 0x562e132082f0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.855+0000 7f194bfff640 1 -- 172.21.15.50:0/524535886 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f195c040070 con 0x562e132082f0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.855+0000 7f194bfff640 1 -- 172.21.15.50:0/524535886 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c003810 con 0x562e132082f0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- 172.21.15.50:0/524535886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 msgr2=0x562e135d4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f19663bf640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 0x7f194c0b4cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x562e135d5960 tx=0x7f1958004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f195c031460 tx=0x7f195c005fd0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- 172.21.15.50:0/524535886 shutdown_connections 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d4cf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 --2- 172.21.15.50:0/524535886 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- 172.21.15.50:0/524535886 >> 172.21.15.50:0/524535886 conn(0x562e13225300 msgr2=0x562e135d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- 172.21.15.50:0/524535886 shutdown_connections 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- 172.21.15.50:0/524535886 wait complete. 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.856+0000 7f196edb2740 1 -- start start 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 Processor -- start 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e135e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e13472ab0 con 0x562e132082f0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e134f34d0 con 0x562e132311b0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e135dc630 con 0x562e132104d0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f196edb2740 10 client.0 _ll_register_callbacks cb 0x562e13531950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47112/0 (socket says 172.21.15.50:47112) 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f1965bbe640 1 -- 172.21.15.50:0/1262363909 learned_addr learned my addr 172.21.15.50:0/1262363909 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f19653bd640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.857+0000 7f19663bf640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e135e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 -- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 msgr2=0x562e135e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e135e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 -- 172.21.15.50:0/1262363909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 msgr2=0x562e135ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 -- 172.21.15.50:0/1262363909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e134f4340 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f1965bbe640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19653bd640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135de4f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19500df820 tx=0x7f19500044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489d6df78cc8c120 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19497fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500a8c40 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f19500e5cf0 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c040070 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19500a8a60 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f196edb2740 10 client.25069 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.858+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e1329c790 con 0x562e132104d0 2024-05-17T04:36:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f195c040070 con 0x562e132104d0 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f19500a5580 con 0x562e132104d0 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 10 client.25069 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 10 client.25069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 20 client.25069 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131efd80 0x562e135eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 4 client.25069 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e13548c10 con 0x562e131efd80 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f1965bbe640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131efd80 0x562e135eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f19497fa640 10 client.25069 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.859+0000 7f1965bbe640 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131efd80 0x562e135eb8d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89dd97c23ee5e0d server_cookie=1f8da9691ca45704 in_seq=0 out_seq=0 2024-05-17T04:36:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f19497fa640 1 -- 172.21.15.50:0/1262363909 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e13548c10 con 0x562e131efd80 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f19497fa640 10 client.25069 handle_command_reply: tid=0 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f196edb2740 1 client.25069 shutdown 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 msgr2=0x562e135de4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135de4f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19500df820 tx=0x7f19500044d0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.860+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 shutdown_connections 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e135e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135de4f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131efd80 0x562e135eb8d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 --2- 172.21.15.50:0/1262363909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f19497fa640 0 client.25069 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 >> 172.21.15.50:0/1262363909 conn(0x562e13225300 msgr2=0x562e135d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 shutdown_connections 2024-05-17T04:36:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 1 -- 172.21.15.50:0/1262363909 wait complete. 2024-05-17T04:36:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.861+0000 7f196edb2740 20 client.25069 trim_cache size 0 max 16384 2024-05-17T04:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.878+0000 7f196edb2740 1 Processor -- start 2024-05-17T04:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.878+0000 7f196edb2740 1 -- start start 2024-05-17T04:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19601b0a80 con 0x562e132082f0 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ae320 con 0x562e132104d0 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f196edb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ac170 con 0x562e132311b0 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47116/0 (socket says 172.21.15.50:47116) 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 -- 172.21.15.50:0/4183331776 learned_addr learned my addr 172.21.15.50:0/4183331776 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f19653bd640 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f19663bf640 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 -- 172.21.15.50:0/4183331776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 msgr2=0x562e135d38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 -- 172.21.15.50:0/4183331776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 msgr2=0x562e135d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 -- 172.21.15.50:0/4183331776 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e13315620 con 0x562e132082f0 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.879+0000 7f1965bbe640 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a38ed8d18c074 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.880+0000 7f194b7fe640 1 -- 172.21.15.50:0/4183331776 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c058070 con 0x562e132082f0 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.880+0000 7f194b7fe640 1 -- 172.21.15.50:0/4183331776 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f195c040050 con 0x562e132082f0 2024-05-17T04:36:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.880+0000 7f194b7fe640 1 -- 172.21.15.50:0/4183331776 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035040 con 0x562e132082f0 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.880+0000 7f196edb2740 1 -- 172.21.15.50:0/4183331776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 msgr2=0x562e135d8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.880+0000 7f196edb2740 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c031bc0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 -- 172.21.15.50:0/4183331776 shutdown_connections 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132311b0 0x562e135d38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132104d0 0x562e135d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 --2- 172.21.15.50:0/4183331776 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135d8a80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 -- 172.21.15.50:0/4183331776 >> 172.21.15.50:0/4183331776 conn(0x562e13225300 msgr2=0x562e135d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 -- 172.21.15.50:0/4183331776 shutdown_connections 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 -- 172.21.15.50:0/4183331776 wait complete. 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 -- start start 2024-05-17T04:36:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.881+0000 7f196edb2740 1 Processor -- start 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135e4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e13651dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ac7e0 con 0x562e132082f0 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19601adc50 con 0x562e132311b0 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19601aceb0 con 0x562e132104d0 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f196edb2740 10 client.0 _ll_register_callbacks cb 0x7f19500046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47132/0 (socket says 172.21.15.50:47132) 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 -- 172.21.15.50:0/352108535 learned_addr learned my addr 172.21.15.50:0/352108535 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f19663bf640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e13651dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f19653bd640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135e4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 -- 172.21.15.50:0/352108535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 msgr2=0x562e135e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135e4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 -- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 msgr2=0x562e13651dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e13651dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 -- 172.21.15.50:0/352108535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e136523b0 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f1965bbe640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195c056040 tx=0x7f195c031590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb30a85ca23a9c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.882+0000 7f19497fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c058070 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f195c053a70 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c040050 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f195c035040 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 10 client.15303 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e13653640 con 0x562e132082f0 2024-05-17T04:36:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f195c040050 con 0x562e132082f0 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f195c03ca50 con 0x562e132082f0 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 10 client.15303 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 20 client.15303 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131c51c0 0x562e13655c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 4 client.15303 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.883+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e135423f0 con 0x562e131c51c0 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.884+0000 7f19653bd640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131c51c0 0x562e13655c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.884+0000 7f19497fa640 10 client.15303 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.884+0000 7f19653bd640 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131c51c0 0x562e13655c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e92d5b279cd84b4 server_cookie=5f85590ad91dc6c2 in_seq=0 out_seq=0 2024-05-17T04:36:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.899+0000 7f19497fa640 1 -- 172.21.15.50:0/352108535 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562e135423f0 con 0x562e131c51c0 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.899+0000 7f19497fa640 10 client.15303 handle_command_reply: tid=0 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 client.15303 shutdown 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 msgr2=0x562e135e3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195c056040 tx=0x7f195c031590 comp rx=0 tx=0).stop 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 shutdown_connections 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e132311b0 0x562e13651dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e132104d0 0x562e135e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e131c51c0 0x562e13655c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 --2- 172.21.15.50:0/352108535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e132082f0 0x562e135e3d30 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f19497fa640 0 client.15303 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 >> 172.21.15.50:0/352108535 conn(0x562e13225300 msgr2=0x562e135d3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 shutdown_connections 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 1 -- 172.21.15.50:0/352108535 wait complete. 2024-05-17T04:36:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.900+0000 7f196edb2740 20 client.15303 trim_cache size 0 max 16384 2024-05-17T04:36:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:36:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:36:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 msgr2=0x7f194c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 0x7f194c0b4cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x562e135d5960 tx=0x7f1958004a40 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 msgr2=0x7f1960075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f195000d950 tx=0x7f195000de10 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 shutdown_connections 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1960104c00 0x7f19600755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f194c0b27e0 0x7f194c0b4cf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f19601042b0 0x7f1960075090 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 --2- 172.21.15.50:0/2098654963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1960103190 0x7f19600769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 >> 172.21.15.50:0/2098654963 conn(0x7f19600fe970 msgr2=0x7f19601075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 shutdown_connections 2024-05-17T04:36:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:33.901+0000 7f1967e22640 1 -- 172.21.15.50:0/2098654963 wait complete. 2024-05-17T04:36:34.224 INFO:teuthology.orchestra.run.smithi050.stderr:2000000 of 3200000 tuples (62%) done (elapsed 21.59 s, remaining 12.95 s) 2024-05-17T04:36:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:33 smithi105 ceph-mon[34795]: pgmap v446: 129 pgs: 129 active+clean; 409 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 63 op/s 2024-05-17T04:36:34.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:33 smithi110 ceph-mon[35497]: pgmap v446: 129 pgs: 129 active+clean; 409 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 63 op/s 2024-05-17T04:36:34.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:33 smithi050 ceph-mon[31739]: pgmap v446: 129 pgs: 129 active+clean; 409 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 63 op/s 2024-05-17T04:36:35.390 INFO:teuthology.orchestra.run.smithi050.stderr:2100000 of 3200000 tuples (65%) done (elapsed 22.76 s, remaining 11.92 s) 2024-05-17T04:36:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:35 smithi105 ceph-mon[34795]: pgmap v447: 129 pgs: 129 active+clean; 477 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 MiB/s wr, 63 op/s 2024-05-17T04:36:36.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:35 smithi110 ceph-mon[35497]: pgmap v447: 129 pgs: 129 active+clean; 477 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 MiB/s wr, 63 op/s 2024-05-17T04:36:36.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:35 smithi050 ceph-mon[31739]: pgmap v447: 129 pgs: 129 active+clean; 477 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 MiB/s wr, 63 op/s 2024-05-17T04:36:36.639 INFO:teuthology.orchestra.run.smithi050.stderr:2200000 of 3200000 tuples (68%) done (elapsed 24.01 s, remaining 10.91 s) 2024-05-17T04:36:37.519 INFO:teuthology.orchestra.run.smithi050.stderr:2300000 of 3200000 tuples (71%) done (elapsed 24.88 s, remaining 9.74 s) 2024-05-17T04:36:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:37 smithi105 ceph-mon[34795]: pgmap v448: 129 pgs: 129 active+clean; 511 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 37 op/s 2024-05-17T04:36:38.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:37 smithi110 ceph-mon[35497]: pgmap v448: 129 pgs: 129 active+clean; 511 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 37 op/s 2024-05-17T04:36:38.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:37 smithi050 ceph-mon[31739]: pgmap v448: 129 pgs: 129 active+clean; 511 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 MiB/s wr, 37 op/s 2024-05-17T04:36:38.781 INFO:teuthology.orchestra.run.smithi050.stderr:2400000 of 3200000 tuples (75%) done (elapsed 26.15 s, remaining 8.72 s) 2024-05-17T04:36:40.054 INFO:teuthology.orchestra.run.smithi050.stderr:2500000 of 3200000 tuples (78%) done (elapsed 27.42 s, remaining 7.68 s) 2024-05-17T04:36:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:39 smithi105 ceph-mon[34795]: pgmap v449: 129 pgs: 129 active+clean; 547 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 36 op/s 2024-05-17T04:36:40.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:39 smithi110 ceph-mon[35497]: pgmap v449: 129 pgs: 129 active+clean; 547 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 36 op/s 2024-05-17T04:36:40.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:39 smithi050 ceph-mon[31739]: pgmap v449: 129 pgs: 129 active+clean; 547 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 36 op/s 2024-05-17T04:36:40.850 INFO:teuthology.orchestra.run.smithi050.stderr:2600000 of 3200000 tuples (81%) done (elapsed 28.22 s, remaining 6.51 s) 2024-05-17T04:36:42.152 INFO:teuthology.orchestra.run.smithi050.stderr:2700000 of 3200000 tuples (84%) done (elapsed 29.52 s, remaining 5.47 s) 2024-05-17T04:36:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:41 smithi105 ceph-mon[34795]: pgmap v450: 129 pgs: 129 active+clean; 583 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 42 op/s 2024-05-17T04:36:42.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:41 smithi110 ceph-mon[35497]: pgmap v450: 129 pgs: 129 active+clean; 583 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 42 op/s 2024-05-17T04:36:42.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:41 smithi050 ceph-mon[31739]: pgmap v450: 129 pgs: 129 active+clean; 583 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 42 MiB/s wr, 42 op/s 2024-05-17T04:36:43.002 INFO:teuthology.orchestra.run.smithi050.stderr:2800000 of 3200000 tuples (87%) done (elapsed 30.37 s, remaining 4.34 s) 2024-05-17T04:36:44.196 INFO:teuthology.orchestra.run.smithi050.stderr:2900000 of 3200000 tuples (90%) done (elapsed 31.56 s, remaining 3.26 s) 2024-05-17T04:36:44.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:43 smithi105 ceph-mon[34795]: pgmap v451: 129 pgs: 129 active+clean; 622 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-17T04:36:44.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:43 smithi110 ceph-mon[35497]: pgmap v451: 129 pgs: 129 active+clean; 622 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-17T04:36:44.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:43 smithi050 ceph-mon[31739]: pgmap v451: 129 pgs: 129 active+clean; 622 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-17T04:36:45.427 INFO:teuthology.orchestra.run.smithi050.stderr:3000000 of 3200000 tuples (93%) done (elapsed 32.79 s, remaining 2.19 s) 2024-05-17T04:36:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:45 smithi105 ceph-mon[34795]: pgmap v452: 129 pgs: 129 active+clean; 662 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 41 op/s 2024-05-17T04:36:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:45 smithi110 ceph-mon[35497]: pgmap v452: 129 pgs: 129 active+clean; 662 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 41 op/s 2024-05-17T04:36:46.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:45 smithi050 ceph-mon[31739]: pgmap v452: 129 pgs: 129 active+clean; 662 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 41 op/s 2024-05-17T04:36:46.326 INFO:teuthology.orchestra.run.smithi050.stderr:3100000 of 3200000 tuples (96%) done (elapsed 33.69 s, remaining 1.09 s) 2024-05-17T04:36:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 -- 172.21.15.50:0/2617565456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094108f30 msgr2=0x7f809410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 --2- 172.21.15.50:0/2617565456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094108f30 0x7f809410b350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f808c009ca0 tx=0x7f808c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 -- 172.21.15.50:0/2617565456 shutdown_connections 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 --2- 172.21.15.50:0/2617565456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094108f30 0x7f809410b350 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 --2- 172.21.15.50:0/2617565456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 0x7f80941089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 --2- 172.21.15.50:0/2617565456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094103c70 0x7f8094106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 -- 172.21.15.50:0/2617565456 >> 172.21.15.50:0/2617565456 conn(0x7f80940fd880 msgr2=0x7f80940ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 -- 172.21.15.50:0/2617565456 shutdown_connections 2024-05-17T04:36:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.015+0000 7f809c59f640 1 -- 172.21.15.50:0/2617565456 wait complete. 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 Processor -- start 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 -- start start 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 0x7f8094199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094108f30 0x7f80941a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f809410eba0 con 0x7f8094103c70 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f809410ea20 con 0x7f80941065d0 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.016+0000 7f809c59f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f809410ed20 con 0x7f8094108f30 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49884/0 (socket says 172.21.15.50:49884) 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 -- 172.21.15.50:0/984045331 learned_addr learned my addr 172.21.15.50:0/984045331 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f8099b3a640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 0x7f8094199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809ab3c640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094108f30 0x7f80941a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 -- 172.21.15.50:0/984045331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094108f30 msgr2=0x7f80941a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094108f30 0x7f80941a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 -- 172.21.15.50:0/984045331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 msgr2=0x7f8094199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 0x7f8094199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 -- 172.21.15.50:0/984045331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80941a1290 con 0x7f8094103c70 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809a33b640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9c0 tx=0x7f809000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f808b7fe640 1 -- 172.21.15.50:0/984045331 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809000cd80 con 0x7f8094103c70 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80941a1580 con 0x7f8094103c70 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f808b7fe640 1 -- 172.21.15.50:0/984045331 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80900044b0 con 0x7f8094103c70 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f808b7fe640 1 -- 172.21.15.50:0/984045331 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090010430 con 0x7f8094103c70 2024-05-17T04:36:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.017+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80941a1b20 con 0x7f8094103c70 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.020+0000 7f80a345e740 1 Processor -- start 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 -- start start 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b7cf16cc0 con 0x559b7cc011b0 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b7cf06860 con 0x559b7cbd82f0 2024-05-17T04:36:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f80a345e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559b7cf0a8a0 con 0x559b7cbe04d0 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f809ab3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f809ab3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49886/0 (socket says 172.21.15.50:49886) 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f809ab3c640 1 -- 172.21.15.50:0/3283916279 learned_addr learned my addr 172.21.15.50:0/3283916279 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f8099b3a640 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.021+0000 7f808b7fe640 1 -- 172.21.15.50:0/984045331 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8090003f00 con 0x7f8094103c70 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 -- 172.21.15.50:0/3283916279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 msgr2=0x559b7cfabc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfabc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 -- 172.21.15.50:0/3283916279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 msgr2=0x559b7cfa4cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 -- 172.21.15.50:0/3283916279 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7ceced50 con 0x559b7cc011b0 2024-05-17T04:36:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f809ab3c640 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f808c009890 tx=0x7f808c0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151ef9abf4b78709 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f8088bfe640 1 -- 172.21.15.50:0/3283916279 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0316d0 con 0x559b7cc011b0 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f8088bfe640 1 -- 172.21.15.50:0/3283916279 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f808c036040 con 0x559b7cc011b0 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f8088bfe640 1 -- 172.21.15.50:0/3283916279 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c004330 con 0x559b7cc011b0 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.022+0000 7f808b7fe640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 0x7f80800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f809a33b640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 0x7f80800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f808b7fe640 1 -- 172.21.15.50:0/984045331 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f80900e2a40 con 0x7f8094103c70 2024-05-17T04:36:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f80a345e740 1 -- 172.21.15.50:0/3283916279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 msgr2=0x559b7cfa4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f809a33b640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 0x7f80800b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80900a5d50 tx=0x7f8090005880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f80a345e740 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f808c009890 tx=0x7f808c0098c0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f80a345e740 1 -- 172.21.15.50:0/3283916279 shutdown_connections 2024-05-17T04:36:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.023+0000 7f80a345e740 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfabc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 --2- 172.21.15.50:0/3283916279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfa4770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- 172.21.15.50:0/3283916279 >> 172.21.15.50:0/3283916279 conn(0x559b7cbf5300 msgr2=0x559b7cfa01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- 172.21.15.50:0/3283916279 shutdown_connections 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- 172.21.15.50:0/3283916279 wait complete. 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- start start 2024-05-17T04:36:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 Processor -- start 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbd82f0 0x559b7cfb2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7cfb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b7ce42ab0 con 0x559b7cbe04d0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b7cec34d0 con 0x559b7cc011b0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559b7cfac630 con 0x559b7cbd82f0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.024+0000 7f80a345e740 10 client.0 _ll_register_callbacks cb 0x559b7cf01950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49892/0 (socket says 172.21.15.50:49892) 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 -- 172.21.15.50:0/4269295778 learned_addr learned my addr 172.21.15.50:0/4269295778 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f809ab3c640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbd82f0 0x559b7cfb2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f809ab3c640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7cfb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 -- 172.21.15.50:0/4269295778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbd82f0 msgr2=0x559b7cfb2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbd82f0 0x559b7cfb2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 -- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 msgr2=0x559b7cfb98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7cfb98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 -- 172.21.15.50:0/4269295778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7cec4340 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f8099b3a640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8084012700 tx=0x7f8084012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4fbaa0b18cb335 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f807dffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004050 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f80840041f0 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.025+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f808c036040 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f80840043f0 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 10 client.15315 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b7cc6c790 con 0x559b7cbe04d0 2024-05-17T04:36:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f808c036040 con 0x559b7cbe04d0 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8084003a80 con 0x559b7cbe04d0 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 10 client.15315 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 20 client.15315 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cbbfd80 0x559b7cfbfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 4 client.15315 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.026+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b7cf18c10 con 0x559b7cbbfd80 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.027+0000 7f809ab3c640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cbbfd80 0x559b7cfbfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.027+0000 7f807dffb640 10 client.15315 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.027+0000 7f809ab3c640 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cbbfd80 0x559b7cfbfd30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7adbee640efe55 server_cookie=e0554c6a7ec64e88 in_seq=0 out_seq=0 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f807dffb640 1 -- 172.21.15.50:0/4269295778 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b7cf18c10 con 0x559b7cbbfd80 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f807dffb640 10 client.15315 handle_command_reply: tid=0 2024-05-17T04:36:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 client.15315 shutdown 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 msgr2=0x559b7cfb2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8084012700 tx=0x7f8084012bc0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 shutdown_connections 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7cfb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cbbfd80 0x559b7cfbfd30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbe04d0 0x559b7cfb2980 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 --2- 172.21.15.50:0/4269295778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbd82f0 0x559b7cfb2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f807dffb640 0 client.15315 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 >> 172.21.15.50:0/4269295778 conn(0x559b7cbf5300 msgr2=0x559b7cf9f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 shutdown_connections 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 1 -- 172.21.15.50:0/4269295778 wait complete. 2024-05-17T04:36:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.028+0000 7f80a345e740 20 client.15315 trim_cache size 0 max 16384 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 Processor -- start 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 -- start start 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfb9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80941ac6f0 con 0x559b7cc011b0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80941a9f90 con 0x559b7cbd82f0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f80a345e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80941a7de0 con 0x559b7cbe04d0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f8099b3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f809ab3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfb9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f8099b3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:38280/0 (socket says 172.21.15.50:38280) 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.051+0000 7f8099b3a640 1 -- 172.21.15.50:0/1700592076 learned_addr learned my addr 172.21.15.50:0/1700592076 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f809ab3c640 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 -- 172.21.15.50:0/1700592076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 msgr2=0x559b7cfa2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 -- 172.21.15.50:0/1700592076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 msgr2=0x559b7cfb9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfb9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 -- 172.21.15.50:0/1700592076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b7cce5620 con 0x559b7cbe04d0 2024-05-17T04:36:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f8099b3a640 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8084003ef0 tx=0x7f8084010810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb3c6a3c8a018e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f807ffff640 1 -- 172.21.15.50:0/1700592076 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808402d020 con 0x559b7cbe04d0 2024-05-17T04:36:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f807ffff640 1 -- 172.21.15.50:0/1700592076 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8084010d90 con 0x559b7cbe04d0 2024-05-17T04:36:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.052+0000 7f807ffff640 1 -- 172.21.15.50:0/1700592076 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808400fce0 con 0x559b7cbe04d0 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 -- 172.21.15.50:0/1700592076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 msgr2=0x559b7cfb9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8084003ef0 tx=0x7f8084010810 comp rx=0 tx=0).stop 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 -- 172.21.15.50:0/1700592076 shutdown_connections 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cc011b0 0x559b7cfb9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb9800 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 --2- 172.21.15.50:0/1700592076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cbd82f0 0x559b7cfa2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 -- 172.21.15.50:0/1700592076 >> 172.21.15.50:0/1700592076 conn(0x559b7cbf5300 msgr2=0x559b7cfaca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 -- 172.21.15.50:0/1700592076 shutdown_connections 2024-05-17T04:36:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.053+0000 7f80a345e740 1 -- 172.21.15.50:0/1700592076 wait complete. 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 -- start start 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 Processor -- start 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7d026090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80941a8450 con 0x559b7cbd82f0 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80941a98c0 con 0x559b7cc011b0 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f80941a8b20 con 0x559b7cbe04d0 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f80a345e740 10 client.0 _ll_register_callbacks cb 0x7f80840097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f809ab3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f809ab3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49918/0 (socket says 172.21.15.50:49918) 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.054+0000 7f809ab3c640 1 -- 172.21.15.50:0/2666067600 learned_addr learned my addr 172.21.15.50:0/2666067600 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:36:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f8099b3a640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7d026090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 -- 172.21.15.50:0/2666067600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 msgr2=0x559b7cfb50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 -- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 msgr2=0x559b7d026090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7d026090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 -- 172.21.15.50:0/2666067600 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7d0265d0 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f809ab3c640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f808c009a60 tx=0x7f808c031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6b00a844468216 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f807dffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c04a040 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f808c004b40 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8084010d90 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f808c065050 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f80a345e740 10 client.15321 fetch_fsmap learned FSMap version 30 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.055+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b7d0276c0 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8084010d90 con 0x559b7cbd82f0 2024-05-17T04:36:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f808c0582d0 con 0x559b7cbd82f0 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 10 client.15321 fetch_fsmap finished waiting for FSMap version 30 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 20 client.15321 populate_metadata read hostname 'smithi050' 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cb951c0 0x559b7d029d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 4 client.15321 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b7cf123f0 con 0x559b7cb951c0 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.056+0000 7f8099b3a640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cb951c0 0x559b7d029d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.057+0000 7f807dffb640 10 client.15321 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:47.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.057+0000 7f8099b3a640 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cb951c0 0x559b7d029d50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a06ff637abdbab server_cookie=53aed272a63c94a in_seq=0 out_seq=0 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.071+0000 7f807dffb640 1 -- 172.21.15.50:0/2666067600 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559b7cf123f0 con 0x559b7cb951c0 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.071+0000 7f807dffb640 10 client.15321 handle_command_reply: tid=0 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.071+0000 7f80a345e740 1 client.15321 shutdown 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 msgr2=0x559b7cfb4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f808c009a60 tx=0x7f808c031980 comp rx=0 tx=0).stop 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 shutdown_connections 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b7cc011b0 0x559b7d026090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b7cbe04d0 0x559b7cfb50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559b7cb951c0 0x559b7d029d50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 --2- 172.21.15.50:0/2666067600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b7cbd82f0 0x559b7cfb4ba0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f807dffb640 0 client.15321 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 >> 172.21.15.50:0/2666067600 conn(0x559b7cbf5300 msgr2=0x559b7cfaca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 shutdown_connections 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 1 -- 172.21.15.50:0/2666067600 wait complete. 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.072+0000 7f80a345e740 20 client.15321 trim_cache size 0 max 16384 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:36:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:36:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:36:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 msgr2=0x7f80800b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 0x7f80800b4cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f80900a5d50 tx=0x7f8090005880 comp rx=0 tx=0).stop 2024-05-17T04:36:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 msgr2=0x7f8094199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f809000e9c0 tx=0x7f809000ee80 comp rx=0 tx=0).stop 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 shutdown_connections 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8094108f30 0x7f80941a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80941065d0 0x7f8094199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80800b27e0 0x7f80800b4cf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 --2- 172.21.15.50:0/984045331 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094103c70 0x7f8094199750 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 >> 172.21.15.50:0/984045331 conn(0x7f80940fd880 msgr2=0x7f8094104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 shutdown_connections 2024-05-17T04:36:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:36:47.073+0000 7f809c59f640 1 -- 172.21.15.50:0/984045331 wait complete. 2024-05-17T04:36:47.603 INFO:teuthology.orchestra.run.smithi050.stderr:3200000 of 3200000 tuples (100%) done (elapsed 34.97 s, remaining 0.00 s) 2024-05-17T04:36:47.693 INFO:teuthology.orchestra.run.smithi050.stderr:vacuuming... 2024-05-17T04:36:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:47 smithi105 ceph-mon[34795]: pgmap v453: 129 pgs: 129 active+clean; 706 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 32 op/s 2024-05-17T04:36:48.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:47 smithi110 ceph-mon[35497]: pgmap v453: 129 pgs: 129 active+clean; 706 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 32 op/s 2024-05-17T04:36:48.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:47 smithi050 ceph-mon[31739]: pgmap v453: 129 pgs: 129 active+clean; 706 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 32 op/s 2024-05-17T04:36:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:49 smithi105 ceph-mon[34795]: pgmap v454: 129 pgs: 129 active+clean; 746 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 MiB/s wr, 34 op/s 2024-05-17T04:36:50.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:49 smithi110 ceph-mon[35497]: pgmap v454: 129 pgs: 129 active+clean; 746 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 MiB/s wr, 34 op/s 2024-05-17T04:36:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:49 smithi050 ceph-mon[31739]: pgmap v454: 129 pgs: 129 active+clean; 746 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 MiB/s wr, 34 op/s 2024-05-17T04:36:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:51 smithi105 ceph-mon[34795]: pgmap v455: 129 pgs: 129 active+clean; 770 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-05-17T04:36:52.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:51 smithi110 ceph-mon[35497]: pgmap v455: 129 pgs: 129 active+clean; 770 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-05-17T04:36:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:51 smithi050 ceph-mon[31739]: pgmap v455: 129 pgs: 129 active+clean; 770 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 40 op/s 2024-05-17T04:36:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:53 smithi105 ceph-mon[34795]: pgmap v456: 129 pgs: 129 active+clean; 776 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 33 op/s 2024-05-17T04:36:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:53 smithi110 ceph-mon[35497]: pgmap v456: 129 pgs: 129 active+clean; 776 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 33 op/s 2024-05-17T04:36:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:53 smithi050 ceph-mon[31739]: pgmap v456: 129 pgs: 129 active+clean; 776 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 33 op/s 2024-05-17T04:36:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:55 smithi105 ceph-mon[34795]: pgmap v457: 129 pgs: 129 active+clean; 776 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 26 MiB/s wr, 76 op/s 2024-05-17T04:36:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:55 smithi110 ceph-mon[35497]: pgmap v457: 129 pgs: 129 active+clean; 776 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 26 MiB/s wr, 76 op/s 2024-05-17T04:36:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:55 smithi050 ceph-mon[31739]: pgmap v457: 129 pgs: 129 active+clean; 776 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 26 MiB/s wr, 76 op/s 2024-05-17T04:36:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:57 smithi105 ceph-mon[34795]: pgmap v458: 129 pgs: 129 active+clean; 776 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 19 MiB/s wr, 113 op/s 2024-05-17T04:36:58.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:57 smithi110 ceph-mon[35497]: pgmap v458: 129 pgs: 129 active+clean; 776 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 19 MiB/s wr, 113 op/s 2024-05-17T04:36:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:57 smithi050 ceph-mon[31739]: pgmap v458: 129 pgs: 129 active+clean; 776 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 19 MiB/s wr, 113 op/s 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/484789370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc103c80 msgr2=0x7fbccc1060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/484789370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc103c80 0x7fbccc1060a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009900 tx=0x7fbcc802a410 comp rx=0 tx=0).stop 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/484789370 shutdown_connections 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/484789370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc108f40 0x7fbccc10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/484789370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc1065e0 0x7fbccc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/484789370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc103c80 0x7fbccc1060a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/484789370 >> 172.21.15.50:0/484789370 conn(0x7fbccc0fd8b0 msgr2=0x7fbccc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.189+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/484789370 shutdown_connections 2024-05-17T04:37:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.190+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/484789370 wait complete. 2024-05-17T04:37:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.190+0000 7fbcd3ea5640 1 Processor -- start 2024-05-17T04:37:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.190+0000 7fbcd3ea5640 1 -- start start 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc103c80 0x7fbccc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc1065e0 0x7fbccc19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10eb60 con 0x7fbccc108f40 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10e9e0 con 0x7fbccc103c80 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd3ea5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10ece0 con 0x7fbccc1065e0 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36646/0 (socket says 172.21.15.50:36646) 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 -- 172.21.15.50:0/1822700818 learned_addr learned my addr 172.21.15.50:0/1822700818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc103c80 0x7fbccc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd1440640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc1065e0 0x7fbccc19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 -- 172.21.15.50:0/1822700818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc1065e0 msgr2=0x7fbccc19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc1065e0 0x7fbccc19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 -- 172.21.15.50:0/1822700818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc103c80 msgr2=0x7fbccc19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc103c80 0x7fbccc19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 -- 172.21.15.50:0/1822700818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1a3540 con 0x7fbccc108f40 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.191+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400eb10 tx=0x7fbcc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.192+0000 7fbcc2ffd640 1 -- 172.21.15.50:0/1822700818 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4004b20 con 0x7fbccc108f40 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.192+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1a3830 con 0x7fbccc108f40 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.192+0000 7fbcc2ffd640 1 -- 172.21.15.50:0/1822700818 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcc4004cc0 con 0x7fbccc108f40 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.192+0000 7fbcc2ffd640 1 -- 172.21.15.50:0/1822700818 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4010690 con 0x7fbccc108f40 2024-05-17T04:37:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.192+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1a3d80 con 0x7fbccc108f40 2024-05-17T04:37:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.195+0000 7fbcc2ffd640 1 -- 172.21.15.50:0/1822700818 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbcc4010830 con 0x7fbccc108f40 2024-05-17T04:37:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.195+0000 7fbcdad66740 1 Processor -- start 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.195+0000 7fbcdad66740 1 -- start start 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a954500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5874d0 0x55937a95b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55937a8bdcc0 con 0x55937a5874d0 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55937a8ad860 con 0x55937a57f2f0 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcdad66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55937a8b18a0 con 0x55937a5a81b0 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:39446/0 (socket says 172.21.15.50:39446) 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 -- 172.21.15.50:0/4131026409 learned_addr learned my addr 172.21.15.50:0/4131026409 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5874d0 0x55937a95b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a954500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 -- 172.21.15.50:0/4131026409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 msgr2=0x55937a954500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a954500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 -- 172.21.15.50:0/4131026409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5874d0 msgr2=0x55937a95b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5874d0 0x55937a95b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcd1440640 1 -- 172.21.15.50:0/4131026409 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937a875d50 con 0x55937a5a81b0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.196+0000 7fbcc2ffd640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 0x7fbcb80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcd1440640 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097e0 tx=0x7fbcbc009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=272ff39edd8732cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcb7fff640 1 -- 172.21.15.50:0/4131026409 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc004030 con 0x55937a5a81b0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcc2ffd640 1 -- 172.21.15.50:0/1822700818 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbcc40e2b50 con 0x7fbccc108f40 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcb7fff640 1 -- 172.21.15.50:0/4131026409 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc00f7c0 con 0x55937a5a81b0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcb7fff640 1 -- 172.21.15.50:0/4131026409 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0038c0 con 0x55937a5a81b0 2024-05-17T04:37:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.197+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 0x7fbcb80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 -- 172.21.15.50:0/4131026409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 msgr2=0x55937a953f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 0x7fbcb80b4cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005ab0 tx=0x7fbcc8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097e0 tx=0x7fbcbc009770 comp rx=0 tx=0).stop 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 -- 172.21.15.50:0/4131026409 shutdown_connections 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5874d0 0x55937a95b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a954500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 --2- 172.21.15.50:0/4131026409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5a81b0 0x55937a953f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.198+0000 7fbcdad66740 1 -- 172.21.15.50:0/4131026409 >> 172.21.15.50:0/4131026409 conn(0x55937a59c300 msgr2=0x55937a9471a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.199+0000 7fbcdad66740 1 -- 172.21.15.50:0/4131026409 shutdown_connections 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.199+0000 7fbcdad66740 1 -- 172.21.15.50:0/4131026409 wait complete. 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.199+0000 7fbcdad66740 1 -- start start 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.199+0000 7fbcdad66740 1 Processor -- start 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a57f2f0 0x55937a95d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a964e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55937a7e9ab0 con 0x55937a5a81b0 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55937a86a4d0 con 0x55937a5874d0 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55937a95bea0 con 0x55937a57f2f0 2024-05-17T04:37:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcdad66740 10 client.0 _ll_register_callbacks cb 0x55937a8a8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56040/0 (socket says 172.21.15.50:56040) 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd1440640 1 -- 172.21.15.50:0/1458036080 learned_addr learned my addr 172.21.15.50:0/1458036080 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd1c41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a57f2f0 0x55937a95d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a964e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 -- 172.21.15.50:0/1458036080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a57f2f0 msgr2=0x55937a95d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a57f2f0 0x55937a95d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 -- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 msgr2=0x55937a95dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.200+0000 7fbcd2442640 1 -- 172.21.15.50:0/1458036080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937a86b340 con 0x55937a5a81b0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcd1440640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcd2442640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a964e00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40a7050 tx=0x7fbcc40a9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25206a3251eabee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:37:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40ad070 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbcc40b29c0 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcbc00f7c0 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbcc40a9de0 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcdad66740 10 client.15333 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55937a613790 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.201+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc00f7c0 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbcc40a5750 con 0x55937a5a81b0 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 10 client.15333 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 20 client.15333 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a566d80 0x55937a96b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 4 client.15333 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:37:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937a8bfc10 con 0x55937a566d80 2024-05-17T04:37:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.202+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a566d80 0x55937a96b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.203+0000 7fbcb57fa640 10 client.15333 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.203+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a566d80 0x55937a96b290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c13be813833c193 server_cookie=2ccc809918ec287a in_seq=0 out_seq=0 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.203+0000 7fbcb57fa640 1 -- 172.21.15.50:0/1458036080 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55937a8bfc10 con 0x55937a566d80 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.203+0000 7fbcb57fa640 10 client.15333 handle_command_reply: tid=0 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.203+0000 7fbcdad66740 1 client.15333 shutdown 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 msgr2=0x55937a964e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a964e00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40a7050 tx=0x7fbcc40a9800 comp rx=0 tx=0).stop 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 shutdown_connections 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a566d80 0x55937a96b290 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a964e00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcb57fa640 0 client.15333 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5874d0 0x55937a95dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 --2- 172.21.15.50:0/1458036080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a57f2f0 0x55937a95d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 >> 172.21.15.50:0/1458036080 conn(0x55937a59c300 msgr2=0x55937a9461a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 shutdown_connections 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 1 -- 172.21.15.50:0/1458036080 wait complete. 2024-05-17T04:37:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.204+0000 7fbcdad66740 20 client.15333 trim_cache size 0 max 16384 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 Processor -- start 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 -- start start 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 0x55937a958490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 0x55937a94a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1ac660 con 0x55937a57f2f0 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a9ed0 con 0x55937a5a81b0 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcdad66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a7d40 con 0x55937a5874d0 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcd1c41640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 0x55937a958490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcd2442640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 0x55937a94a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:39464/0 (socket says 172.21.15.50:39464) 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.227+0000 7fbcd1440640 1 -- 172.21.15.50:0/3674658027 learned_addr learned my addr 172.21.15.50:0/3674658027 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 -- 172.21.15.50:0/3674658027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 msgr2=0x55937a94a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 0x55937a94a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 -- 172.21.15.50:0/3674658027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 msgr2=0x55937a958490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 0x55937a958490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 -- 172.21.15.50:0/3674658027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937a68c620 con 0x55937a5874d0 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd2442640 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 0x55937a94a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 0x55937a958490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcd1440640 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0095c0 tx=0x7fbcbc009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4c3666cbe246c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcb77fe640 1 -- 172.21.15.50:0/3674658027 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc02d070 con 0x55937a5874d0 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcb77fe640 1 -- 172.21.15.50:0/3674658027 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc00f790 con 0x55937a5874d0 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.228+0000 7fbcb77fe640 1 -- 172.21.15.50:0/3674658027 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc003830 con 0x55937a5874d0 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 -- 172.21.15.50:0/3674658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 msgr2=0x55937a964d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0095c0 tx=0x7fbcbc009c70 comp rx=0 tx=0).stop 2024-05-17T04:37:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 -- 172.21.15.50:0/3674658027 shutdown_connections 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a5a81b0 0x55937a94a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a964d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 --2- 172.21.15.50:0/3674658027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a57f2f0 0x55937a958490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.229+0000 7fbcdad66740 1 -- 172.21.15.50:0/3674658027 >> 172.21.15.50:0/3674658027 conn(0x55937a59c300 msgr2=0x55937a949ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 -- 172.21.15.50:0/3674658027 shutdown_connections 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 -- 172.21.15.50:0/3674658027 wait complete. 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 -- start start 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 Processor -- start 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a9621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.230+0000 7fbcdad66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcdad66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a8390 con 0x55937a5a81b0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcdad66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a9800 con 0x55937a57f2f0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcdad66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1a8a60 con 0x55937a5874d0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcdad66740 10 client.0 _ll_register_callbacks cb 0x7fbcc4005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd2442640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:39476/0 (socket says 172.21.15.50:39476) 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 -- 172.21.15.50:0/388875237 learned_addr learned my addr 172.21.15.50:0/388875237 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd2442640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36668/0 (socket says 172.21.15.50:36668) 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a9621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 -- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 msgr2=0x55937a9621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a9621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 -- 172.21.15.50:0/388875237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 msgr2=0x55937a9635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 -- 172.21.15.50:0/388875237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937a963be0 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd2442640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcd1440640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc011660 tx=0x7fbcbc0054b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6544195d1ed7f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.231+0000 7fbcb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc036060 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcbc00f790 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbcbc025040 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc03d050 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 10 client.25111 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55937a964e70 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcbc00f790 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fbcbc028530 con 0x55937a5874d0 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 10 client.25111 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 10 client.25111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 20 client.25111 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a53c1c0 0x55937a9d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 4 client.25111 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.232+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55937a8b93f0 con 0x55937a53c1c0 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.233+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a53c1c0 0x55937a9d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.233+0000 7fbcb57fa640 10 client.25111 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.233+0000 7fbcd1c41640 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a53c1c0 0x55937a9d5980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f521335558e164cb server_cookie=efb023d4eb442291 in_seq=0 out_seq=0 2024-05-17T04:37:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcb57fa640 1 -- 172.21.15.50:0/388875237 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55937a8b93f0 con 0x55937a53c1c0 2024-05-17T04:37:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcb57fa640 10 client.25111 handle_command_reply: tid=0 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 client.25111 shutdown 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 msgr2=0x55937a962730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc011660 tx=0x7fbcbc0054b0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 shutdown_connections 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55937a53c1c0 0x55937a9d5980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55937a5a81b0 0x55937a9635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcb57fa640 0 client.25111 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55937a5874d0 0x55937a962730 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 --2- 172.21.15.50:0/388875237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55937a57f2f0 0x55937a9621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 >> 172.21.15.50:0/388875237 conn(0x55937a59c300 msgr2=0x55937a94a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 shutdown_connections 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.252+0000 7fbcdad66740 1 -- 172.21.15.50:0/388875237 wait complete. 2024-05-17T04:37:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcdad66740 20 client.25111 trim_cache size 0 max 16384 2024-05-17T04:37:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:37:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:37:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 msgr2=0x7fbcb80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 0x7fbcb80b4cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005ab0 tx=0x7fbcc8000bc0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 msgr2=0x7fbccc1a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400eb10 tx=0x7fbcc400c2d0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 shutdown_connections 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbcb80b27e0 0x7fbcb80b4cf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbccc108f40 0x7fbccc1a2ed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbccc1065e0 0x7fbccc19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 --2- 172.21.15.50:0/1822700818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbccc103c80 0x7fbccc19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.253+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 >> 172.21.15.50:0/1822700818 conn(0x7fbccc0fd8b0 msgr2=0x7fbccc101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.254+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 shutdown_connections 2024-05-17T04:37:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:00.254+0000 7fbcd3ea5640 1 -- 172.21.15.50:0/1822700818 wait complete. 2024-05-17T04:37:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:59 smithi105 ceph-mon[34795]: pgmap v459: 129 pgs: 129 active+clean; 776 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 164 op/s 2024-05-17T04:37:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:36:59 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:37:00.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:59 smithi110 ceph-mon[35497]: pgmap v459: 129 pgs: 129 active+clean; 776 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 164 op/s 2024-05-17T04:37:00.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:36:59 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:37:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:59 smithi050 ceph-mon[31739]: pgmap v459: 129 pgs: 129 active+clean; 776 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 12 MiB/s wr, 164 op/s 2024-05-17T04:37:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:36:59 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:37:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:01 smithi105 ceph-mon[34795]: pgmap v460: 129 pgs: 129 active+clean; 776 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.9 MiB/s wr, 209 op/s 2024-05-17T04:37:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:01 smithi110 ceph-mon[35497]: pgmap v460: 129 pgs: 129 active+clean; 776 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.9 MiB/s wr, 209 op/s 2024-05-17T04:37:02.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:01 smithi050 ceph-mon[31739]: pgmap v460: 129 pgs: 129 active+clean; 776 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.9 MiB/s wr, 209 op/s 2024-05-17T04:37:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:03 smithi105 ceph-mon[34795]: pgmap v461: 129 pgs: 129 active+clean; 776 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-17T04:37:04.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:03 smithi110 ceph-mon[35497]: pgmap v461: 129 pgs: 129 active+clean; 776 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-17T04:37:04.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:03 smithi050 ceph-mon[31739]: pgmap v461: 129 pgs: 129 active+clean; 776 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-17T04:37:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:05 smithi105 ceph-mon[34795]: pgmap v462: 129 pgs: 129 active+clean; 776 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 478 op/s 2024-05-17T04:37:06.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:05 smithi110 ceph-mon[35497]: pgmap v462: 129 pgs: 129 active+clean; 776 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 478 op/s 2024-05-17T04:37:06.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:05 smithi050 ceph-mon[31739]: pgmap v462: 129 pgs: 129 active+clean; 776 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 478 op/s 2024-05-17T04:37:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:07 smithi105 ceph-mon[34795]: pgmap v463: 129 pgs: 129 active+clean; 776 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-17T04:37:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:07 smithi110 ceph-mon[35497]: pgmap v463: 129 pgs: 129 active+clean; 776 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-17T04:37:08.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:07 smithi050 ceph-mon[31739]: pgmap v463: 129 pgs: 129 active+clean; 776 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 479 op/s 2024-05-17T04:37:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:09 smithi105 ceph-mon[34795]: pgmap v464: 129 pgs: 129 active+clean; 776 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:09 smithi110 ceph-mon[35497]: pgmap v464: 129 pgs: 129 active+clean; 776 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:10.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:09 smithi050 ceph-mon[31739]: pgmap v464: 129 pgs: 129 active+clean; 776 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:11 smithi105 ceph-mon[34795]: pgmap v465: 129 pgs: 129 active+clean; 776 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-05-17T04:37:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:11 smithi110 ceph-mon[35497]: pgmap v465: 129 pgs: 129 active+clean; 776 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-05-17T04:37:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:11 smithi050 ceph-mon[31739]: pgmap v465: 129 pgs: 129 active+clean; 776 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 535 op/s 2024-05-17T04:37:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 -- 172.21.15.50:0/1456293748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6078103af0 msgr2=0x7f607810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 --2- 172.21.15.50:0/1456293748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6078103af0 0x7f607810a3f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6070009ca0 tx=0x7f607002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 -- 172.21.15.50:0/1456293748 shutdown_connections 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 --2- 172.21.15.50:0/1456293748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6078103af0 0x7f607810a3f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 --2- 172.21.15.50:0/1456293748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60781031a0 0x7f60781035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 --2- 172.21.15.50:0/1456293748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078102080 0x7f6078102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 -- 172.21.15.50:0/1456293748 >> 172.21.15.50:0/1456293748 conn(0x7f60780fd880 msgr2=0x7f60780ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 -- 172.21.15.50:0/1456293748 shutdown_connections 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.360+0000 7f607e3ac640 1 -- 172.21.15.50:0/1456293748 wait complete. 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 Processor -- start 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 -- start start 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6078102080 0x7f607806b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 0x7f607819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f607810da10 con 0x7f60781031a0 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f607810d890 con 0x7f6078103af0 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607e3ac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f607810db90 con 0x7f6078102080 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f606ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 0x7f607819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607c949640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607c949640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:47500/0 (socket says 172.21.15.50:47500) 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f6077fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6078102080 0x7f607806b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f607c949640 1 -- 172.21.15.50:0/2340065731 learned_addr learned my addr 172.21.15.50:0/2340065731 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.361+0000 7f606ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 0x7f607819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53446/0 (socket says 172.21.15.50:53446) 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 -- 172.21.15.50:0/2340065731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6078102080 msgr2=0x7f607806b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6078102080 0x7f607806b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 -- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 msgr2=0x7f607819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 0x7f607819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 -- 172.21.15.50:0/2340065731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60781a12e0 con 0x7f6078103af0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607c949640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6070009dc0 tx=0x7f6070003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f6075ffb640 1 -- 172.21.15.50:0/2340065731 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070004240 con 0x7f6078103af0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6078198300 con 0x7f6078103af0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60781988a0 con 0x7f6078103af0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f6075ffb640 1 -- 172.21.15.50:0/2340065731 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6070040030 con 0x7f6078103af0 2024-05-17T04:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.362+0000 7f6075ffb640 1 -- 172.21.15.50:0/2340065731 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070030cf0 con 0x7f6078103af0 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.364+0000 7f608525f740 1 Processor -- start 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f6075ffb640 1 -- 172.21.15.50:0/2340065731 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f60700043e0 con 0x7f6078103af0 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 -- start start 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 0x55bbc5d2dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5c9fcc0 con 0x55bbc598a1b0 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5c8f860 con 0x55bbc59694d0 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.365+0000 7f608525f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5c938a0 con 0x55bbc59612f0 2024-05-17T04:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53460/0 (socket says 172.21.15.50:53460) 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f6077fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 0x55bbc5d2dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 -- 172.21.15.50:0/1308933560 learned_addr learned my addr 172.21.15.50:0/1308933560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 -- 172.21.15.50:0/1308933560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 msgr2=0x55bbc5d2dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 0x55bbc5d2dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 -- 172.21.15.50:0/1308933560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 msgr2=0x55bbc5d34c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d34c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 -- 172.21.15.50:0/1308933560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc5c57d50 con 0x55bbc598a1b0 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f6077fff640 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 0x55bbc5d2dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f6075ffb640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 0x7f60640b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606ffff640 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6060009a40 tx=0x7f6060005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b23241cb3bbaf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f6077fff640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 0x7f60640b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f6075ffb640 1 -- 172.21.15.50:0/2340065731 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6070103260 con 0x7f6078103af0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606e7fc640 1 -- 172.21.15.50:0/1308933560 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060004030 con 0x55bbc598a1b0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606e7fc640 1 -- 172.21.15.50:0/1308933560 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606000b780 con 0x55bbc598a1b0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.366+0000 7f606e7fc640 1 -- 172.21.15.50:0/1308933560 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600035f0 con 0x55bbc598a1b0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 -- 172.21.15.50:0/1308933560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 msgr2=0x55bbc5d2d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f6077fff640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 0x7f60640b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55bbc5d2eb70 tx=0x7f6068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6060009a40 tx=0x7f6060005f90 comp rx=0 tx=0).stop 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 -- 172.21.15.50:0/1308933560 shutdown_connections 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d34c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59612f0 0x55bbc5d2dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 --2- 172.21.15.50:0/1308933560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc598a1b0 0x55bbc5d2d770 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 -- 172.21.15.50:0/1308933560 >> 172.21.15.50:0/1308933560 conn(0x55bbc597e300 msgr2=0x55bbc5d291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 -- 172.21.15.50:0/1308933560 shutdown_connections 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.367+0000 7f608525f740 1 -- 172.21.15.50:0/1308933560 wait complete. 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 -- start start 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 Processor -- start 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d37660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5bcbab0 con 0x55bbc59612f0 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5c4c4d0 con 0x55bbc598a1b0 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bbc5d35690 con 0x55bbc59694d0 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f608525f740 10 client.0 _ll_register_callbacks cb 0x55bbc5c8a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53472/0 (socket says 172.21.15.50:53472) 2024-05-17T04:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f6077fff640 1 -- 172.21.15.50:0/2199094806 learned_addr learned my addr 172.21.15.50:0/2199094806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f606ffff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d37660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.368+0000 7f607c949640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 -- 172.21.15.50:0/2199094806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 msgr2=0x55bbc5d37660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d37660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 -- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 msgr2=0x55bbc5d3e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 -- 172.21.15.50:0/2199094806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc5c4d340 con 0x55bbc59612f0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f6077fff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6068013010 tx=0x7f6068004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32dc84418df99e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f60577fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606801f640 con 0x55bbc59612f0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f606801f7e0 con 0x55bbc59612f0 2024-05-17T04:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606000b780 con 0x55bbc59612f0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f606801f9e0 con 0x55bbc59612f0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f608525f740 10 client.15351 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.369+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bbc59f5790 con 0x55bbc59612f0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606000b780 con 0x55bbc59612f0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6068027070 con 0x55bbc59612f0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 10 client.15351 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 20 client.15351 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc5948d80 0x55bbc5d449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 4 client.15351 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbc5ca1c10 con 0x55bbc5948d80 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.370+0000 7f606ffff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc5948d80 0x55bbc5d449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.371+0000 7f606ffff640 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc5948d80 0x55bbc5d449f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8010e0de27f4e914 server_cookie=4ab2be978f33f183 in_seq=0 out_seq=0 2024-05-17T04:37:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.371+0000 7f60577fe640 10 client.15351 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.371+0000 7f60577fe640 1 -- 172.21.15.50:0/2199094806 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbc5ca1c10 con 0x55bbc5948d80 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.371+0000 7f60577fe640 10 client.15351 handle_command_reply: tid=0 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.371+0000 7f608525f740 1 client.15351 shutdown 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 msgr2=0x55bbc5d37120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6068013010 tx=0x7f6068004730 comp rx=0 tx=0).stop 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 shutdown_connections 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d37660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc5948d80 0x55bbc5d449f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 --2- 172.21.15.50:0/2199094806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d37120 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f60577fe640 0 client.15351 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 >> 172.21.15.50:0/2199094806 conn(0x55bbc597e300 msgr2=0x55bbc5d2c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 shutdown_connections 2024-05-17T04:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 1 -- 172.21.15.50:0/2199094806 wait complete. 2024-05-17T04:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.372+0000 7f608525f740 20 client.15351 trim_cache size 0 max 16384 2024-05-17T04:37:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 Processor -- start 2024-05-17T04:37:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 -- start start 2024-05-17T04:37:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d3e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60781ac720 con 0x55bbc59612f0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60781a9f90 con 0x55bbc59694d0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f608525f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60781a7de0 con 0x55bbc598a1b0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53476/0 (socket says 172.21.15.50:53476) 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f6077fff640 1 -- 172.21.15.50:0/1818015801 learned_addr learned my addr 172.21.15.50:0/1818015801 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.388+0000 7f606ffff640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d3e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f607c949640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 -- 172.21.15.50:0/1818015801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 msgr2=0x55bbc5d3ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 -- 172.21.15.50:0/1818015801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 msgr2=0x55bbc5d3e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d3e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 -- 172.21.15.50:0/1818015801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc5a6e620 con 0x55bbc59612f0 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f607c949640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f6077fff640 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f606800a010 tx=0x7f60680044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955bb341993eba2e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f606dffb640 1 -- 172.21.15.50:0/1818015801 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606801f640 con 0x55bbc59612f0 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f606dffb640 1 -- 172.21.15.50:0/1818015801 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606801f7e0 con 0x55bbc59612f0 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.389+0000 7f606dffb640 1 -- 172.21.15.50:0/1818015801 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606801d900 con 0x55bbc59612f0 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- 172.21.15.50:0/1818015801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 msgr2=0x55bbc5d2c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f606800a010 tx=0x7f60680044e0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- 172.21.15.50:0/1818015801 shutdown_connections 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc59694d0 0x55bbc5d3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 --2- 172.21.15.50:0/1818015801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d2c5b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- 172.21.15.50:0/1818015801 >> 172.21.15.50:0/1818015801 conn(0x55bbc597e300 msgr2=0x55bbc5d2bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- 172.21.15.50:0/1818015801 shutdown_connections 2024-05-17T04:37:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- 172.21.15.50:0/1818015801 wait complete. 2024-05-17T04:37:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.390+0000 7f608525f740 1 -- start start 2024-05-17T04:37:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 Processor -- start 2024-05-17T04:37:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d3c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60781a8450 con 0x55bbc59612f0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60781a98c0 con 0x55bbc598a1b0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60781a8b20 con 0x55bbc59694d0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f608525f740 10 client.0 _ll_register_callbacks cb 0x7f6068009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53478/0 (socket says 172.21.15.50:53478) 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 -- 172.21.15.50:0/675729553 learned_addr learned my addr 172.21.15.50:0/675729553 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f607c949640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f606ffff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d3c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 -- 172.21.15.50:0/675729553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 msgr2=0x55bbc5d3c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d3c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 -- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 msgr2=0x55bbc5d3d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.391+0000 7f6077fff640 1 -- 172.21.15.50:0/675729553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc5d3d940 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f6077fff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6068012d20 tx=0x7f606801fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669edeb1f5e326f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068046060 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6068029850 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606801f7e0 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f606801f640 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 10 client.15357 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bbc5d3ebd0 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f606801f7e0 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6068027070 con 0x55bbc59612f0 2024-05-17T04:37:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 10 client.15357 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 20 client.15357 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.392+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc591e1c0 0x55bbc5daee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.393+0000 7f608525f740 4 client.15357 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.393+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbc5c9b3f0 con 0x55bbc591e1c0 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.393+0000 7f606ffff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc591e1c0 0x55bbc5daee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.393+0000 7f60577fe640 10 client.15357 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.393+0000 7f606ffff640 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc591e1c0 0x55bbc5daee70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebbf67e9978449f3 server_cookie=31be717ad51f74e1 in_seq=0 out_seq=0 2024-05-17T04:37:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f60577fe640 1 -- 172.21.15.50:0/675729553 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bbc5c9b3f0 con 0x55bbc591e1c0 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f60577fe640 10 client.15357 handle_command_reply: tid=0 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 client.15357 shutdown 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 msgr2=0x55bbc5d3bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6068012d20 tx=0x7f606801fe90 comp rx=0 tx=0).stop 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 shutdown_connections 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbc598a1b0 0x55bbc5d3d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bbc59694d0 0x55bbc5d3c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bbc591e1c0 0x55bbc5daee70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 --2- 172.21.15.50:0/675729553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bbc59612f0 0x55bbc5d3bee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f60577fe640 0 client.15357 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 >> 172.21.15.50:0/675729553 conn(0x55bbc597e300 msgr2=0x55bbc5d2c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 shutdown_connections 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.410+0000 7f608525f740 1 -- 172.21.15.50:0/675729553 wait complete. 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f608525f740 20 client.15357 trim_cache size 0 max 16384 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 msgr2=0x7f60640b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 0x7f60640b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55bbc5d2eb70 tx=0x7f6068005f90 comp rx=0 tx=0).stop 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 msgr2=0x7f60781a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6070009dc0 tx=0x7f6070003ba0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 shutdown_connections 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6078103af0 0x7f60781a0ac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f60640b27e0 0x7f60640b4cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60781031a0 0x7f607819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 --2- 172.21.15.50:0/2340065731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6078102080 0x7f607806b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 >> 172.21.15.50:0/2340065731 conn(0x7f60780fd880 msgr2=0x7f6078106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 shutdown_connections 2024-05-17T04:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:13.411+0000 7f607e3ac640 1 -- 172.21.15.50:0/2340065731 wait complete. 2024-05-17T04:37:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:13 smithi105 ceph-mon[34795]: pgmap v466: 129 pgs: 129 active+clean; 776 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-17T04:37:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:13 smithi110 ceph-mon[35497]: pgmap v466: 129 pgs: 129 active+clean; 776 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-17T04:37:14.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:13 smithi050 ceph-mon[31739]: pgmap v466: 129 pgs: 129 active+clean; 776 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-17T04:37:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:15 smithi105 ceph-mon[34795]: pgmap v467: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-05-17T04:37:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:15 smithi110 ceph-mon[35497]: pgmap v467: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-05-17T04:37:16.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:15 smithi050 ceph-mon[31739]: pgmap v467: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-05-17T04:37:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:17 smithi105 ceph-mon[34795]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-05-17T04:37:18.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:17 smithi110 ceph-mon[35497]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-05-17T04:37:18.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:17 smithi050 ceph-mon[31739]: pgmap v468: 129 pgs: 129 active+clean; 776 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-05-17T04:37:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:19 smithi105 ceph-mon[34795]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 557 op/s 2024-05-17T04:37:20.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:19 smithi110 ceph-mon[35497]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 557 op/s 2024-05-17T04:37:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:19 smithi050 ceph-mon[31739]: pgmap v469: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 557 op/s 2024-05-17T04:37:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:21 smithi105 ceph-mon[34795]: pgmap v470: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-17T04:37:22.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:21 smithi110 ceph-mon[35497]: pgmap v470: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-17T04:37:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:21 smithi050 ceph-mon[31739]: pgmap v470: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 600 op/s 2024-05-17T04:37:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:24 smithi105 ceph-mon[34795]: pgmap v471: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:37:24.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:24 smithi110 ceph-mon[35497]: pgmap v471: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:37:24.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:23 smithi050 ceph-mon[31739]: pgmap v471: 129 pgs: 129 active+clean; 776 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:37:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:26 smithi105 ceph-mon[34795]: pgmap v472: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-05-17T04:37:26.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:26 smithi110 ceph-mon[35497]: pgmap v472: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-05-17T04:37:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:26 smithi050 ceph-mon[31739]: pgmap v472: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-05-17T04:37:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 -- 172.21.15.50:0/2828908766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 msgr2=0x7fe1b4105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 --2- 172.21.15.50:0/2828908766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b4105ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009900 tx=0x7fe1a402a410 comp rx=0 tx=0).stop 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 -- 172.21.15.50:0/2828908766 shutdown_connections 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 --2- 172.21.15.50:0/2828908766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 0x7fe1b410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 --2- 172.21.15.50:0/2828908766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 --2- 172.21.15.50:0/2828908766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b4105ce0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 -- 172.21.15.50:0/2828908766 >> 172.21.15.50:0/2828908766 conn(0x7fe1b40fd880 msgr2=0x7fe1b40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 -- 172.21.15.50:0/2828908766 shutdown_connections 2024-05-17T04:37:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.531+0000 7fe1ba132640 1 -- 172.21.15.50:0/2828908766 wait complete. 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.532+0000 7fe1ba132640 1 Processor -- start 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.532+0000 7fe1ba132640 1 -- start start 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 0x7fe1b41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410eb50 con 0x7fe1b406b230 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410e9d0 con 0x7fe1b4108e20 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1ba132640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410ecd0 con 0x7fe1b41062b0 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57210/0 (socket says 172.21.15.50:57210) 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b37fe640 1 -- 172.21.15.50:0/1130693077 learned_addr learned my addr 172.21.15.50:0/1130693077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 0x7fe1b41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1130693077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 msgr2=0x7fe1b41a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 0x7fe1b41a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 msgr2=0x7fe1b41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1130693077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b41a1320 con 0x7fe1b41062b0 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.533+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4199d20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9f0 tx=0x7fe1a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1b0ff9640 1 -- 172.21.15.50:0/1130693077 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800ccc0 con 0x7fe1b41062b0 2024-05-17T04:37:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1b0ff9640 1 -- 172.21.15.50:0/1130693077 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1a80108c0 con 0x7fe1b41062b0 2024-05-17T04:37:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b41a1610 con 0x7fe1b41062b0 2024-05-17T04:37:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1b0ff9640 1 -- 172.21.15.50:0/1130693077 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80187e0 con 0x7fe1b41062b0 2024-05-17T04:37:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.534+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b41a1b60 con 0x7fe1b41062b0 2024-05-17T04:37:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.537+0000 7fe1c0fea740 1 Processor -- start 2024-05-17T04:37:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.537+0000 7fe1c0fea740 1 -- start start 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.537+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.537+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08972f0 0x5585d0c63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.537+0000 7fe1b0ff9640 1 -- 172.21.15.50:0/1130693077 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe1a8018980 con 0x7fe1b41062b0 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d089f4d0 0x5585d0c6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5585d0bd5cc0 con 0x5585d089f4d0 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5585d0bc5860 con 0x5585d08972f0 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5585d0bc98a0 con 0x5585d08c01b0 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48628/0 (socket says 172.21.15.50:48628) 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 -- 172.21.15.50:0/1845239666 learned_addr learned my addr 172.21.15.50:0/1845239666 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d089f4d0 0x5585d0c6ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08972f0 0x5585d0c63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 -- 172.21.15.50:0/1845239666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08972f0 msgr2=0x5585d0c63cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08972f0 0x5585d0c63cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 -- 172.21.15.50:0/1845239666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d089f4d0 msgr2=0x5585d0c6ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d089f4d0 0x5585d0c6ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 -- 172.21.15.50:0/1845239666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d0b8dd50 con 0x5585d08c01b0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.538+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009fd0 tx=0x7fe1ac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29f41a1989eb4634 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe199ffb640 1 -- 172.21.15.50:0/1845239666 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004d20 con 0x5585d08c01b0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe199ffb640 1 -- 172.21.15.50:0/1845239666 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1ac004050 con 0x5585d08c01b0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe199ffb640 1 -- 172.21.15.50:0/1845239666 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac005220 con 0x5585d08c01b0 2024-05-17T04:37:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe1b0ff9640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 0x7fe19c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 0x7fe19c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.539+0000 7fe1b0ff9640 1 -- 172.21.15.50:0/1130693077 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1a80e28e0 con 0x7fe1b41062b0 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1845239666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 msgr2=0x5585d0c63770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 0x7fe19c0b4cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000c00 tx=0x7fe1a40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009fd0 tx=0x7fe1ac0118d0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1845239666 shutdown_connections 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d089f4d0 0x5585d0c6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08972f0 0x5585d0c63cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1845239666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0c63770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1845239666 >> 172.21.15.50:0/1845239666 conn(0x5585d08b4300 msgr2=0x5585d0c5f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1845239666 shutdown_connections 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1845239666 wait complete. 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.540+0000 7fe1c0fea740 1 -- start start 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 Processor -- start 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c71980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5585d0b01ab0 con 0x5585d08972f0 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5585d0b824d0 con 0x5585d08c01b0 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5585d0c6b690 con 0x5585d089f4d0 2024-05-17T04:37:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1c0fea740 10 client.0 _ll_register_callbacks cb 0x5585d0bc0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57232/0 (socket says 172.21.15.50:57232) 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1b37fe640 1 -- 172.21.15.50:0/93043940 learned_addr learned my addr 172.21.15.50:0/93043940 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.541+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c71980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 -- 172.21.15.50:0/93043940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 msgr2=0x5585d0c71980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c71980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 -- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 msgr2=0x5585d0c788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 -- 172.21.15.50:0/93043940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d0b83340 con 0x5585d08972f0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009230 tx=0x7fe1a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a5ea1460c13ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4035040 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1a40035f0 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ac004050 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe1a40044c0 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1c0fea740 10 client.15369 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.542+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5585d092b790 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1ac004050 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe1a4044070 con 0x5585d08972f0 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 10 client.15369 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 10 client.15369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 20 client.15369 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d087ed80 0x5585d0c7ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 4 client.15369 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585d0bd7c10 con 0x5585d087ed80 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.543+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d087ed80 0x5585d0c7ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.544+0000 7fe192ffd640 10 client.15369 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.544+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d087ed80 0x5585d0c7ec40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43624942c620b01 server_cookie=a3e7b8a6747b48f2 in_seq=0 out_seq=0 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.544+0000 7fe192ffd640 1 -- 172.21.15.50:0/93043940 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5585d0bd7c10 con 0x5585d087ed80 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.544+0000 7fe192ffd640 10 client.15369 handle_command_reply: tid=0 2024-05-17T04:37:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.544+0000 7fe1c0fea740 1 client.15369 shutdown 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 msgr2=0x5585d0c71440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009230 tx=0x7fe1a4009260 comp rx=0 tx=0).stop 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 shutdown_connections 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c71980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d087ed80 0x5585d0c7ec40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/93043940 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c71440 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 >> 172.21.15.50:0/93043940 conn(0x5585d08b4300 msgr2=0x5585d0c62010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe192ffd640 0 client.15369 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 shutdown_connections 2024-05-17T04:37:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 1 -- 172.21.15.50:0/93043940 wait complete. 2024-05-17T04:37:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.545+0000 7fe1c0fea740 20 client.15369 trim_cache size 0 max 16384 2024-05-17T04:37:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.564+0000 7fe1c0fea740 1 Processor -- start 2024-05-17T04:37:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.564+0000 7fe1c0fea740 1 -- start start 2024-05-17T04:37:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c78870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c78db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41ac780 con 0x5585d08972f0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41a9ff0 con 0x5585d08c01b0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1c0fea740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41a7e40 con 0x5585d089f4d0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57246/0 (socket says 172.21.15.50:57246) 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c78870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 -- 172.21.15.50:0/943016243 learned_addr learned my addr 172.21.15.50:0/943016243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c78db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 -- 172.21.15.50:0/943016243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 msgr2=0x5585d0c78870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c78870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 -- 172.21.15.50:0/943016243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 msgr2=0x5585d0c78db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c78db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 -- 172.21.15.50:0/943016243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585d09a4620 con 0x5585d08972f0 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.565+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1a403da60 tx=0x7fe1a4004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ac2105e0a9f67f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.566+0000 7fe1997fa640 1 -- 172.21.15.50:0/943016243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4035040 con 0x5585d08972f0 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.566+0000 7fe1997fa640 1 -- 172.21.15.50:0/943016243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1a403d650 con 0x5585d08972f0 2024-05-17T04:37:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.566+0000 7fe1997fa640 1 -- 172.21.15.50:0/943016243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4003ea0 con 0x5585d08972f0 2024-05-17T04:37:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- 172.21.15.50:0/943016243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 msgr2=0x5585d0c679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1a403da60 tx=0x7fe1a4004450 comp rx=0 tx=0).stop 2024-05-17T04:37:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- 172.21.15.50:0/943016243 shutdown_connections 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d08c01b0 0x5585d0c78db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d089f4d0 0x5585d0c78870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/943016243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c679b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- 172.21.15.50:0/943016243 >> 172.21.15.50:0/943016243 conn(0x5585d08b4300 msgr2=0x5585d0c62510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- 172.21.15.50:0/943016243 shutdown_connections 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- 172.21.15.50:0/943016243 wait complete. 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 -- start start 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.567+0000 7fe1c0fea740 1 Processor -- start 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 0x5585d0c782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b37fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c77d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57260/0 (socket says 172.21.15.50:57260) 2024-05-17T04:37:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0ce4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b37fe640 1 -- 172.21.15.50:0/1178303316 learned_addr learned my addr 172.21.15.50:0/1178303316 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41a84b0 con 0x5585d08972f0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41a9920 con 0x5585d089f4d0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b41a8b80 con 0x5585d08c01b0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 0x5585d0c782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1c0fea740 10 client.0 _ll_register_callbacks cb 0x7fe1a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0ce4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1178303316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 msgr2=0x5585d0ce4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0ce4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1178303316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 msgr2=0x5585d0c77d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c77d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585d0ce5520 con 0x5585d089f4d0 2024-05-17T04:37:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b3fff640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0ce4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe1b2ffd640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 0x5585d0c782a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80e00a0 tx=0x7fe1a80cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8416e280e4878dad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.568+0000 7fe192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80a53b0 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1a8010430 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a403d650 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe1a80b2310 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 10 client.25157 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5585d0ce66f0 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1a403d650 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe1a80c40a0 con 0x5585d089f4d0 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 10 client.25157 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 10 client.25157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 20 client.25157 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d08541c0 0x5585d0ce9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 4 client.25157 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.569+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585d0bd13f0 con 0x5585d08541c0 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.570+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d08541c0 0x5585d0ce9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.570+0000 7fe192ffd640 10 client.25157 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.570+0000 7fe1b37fe640 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d08541c0 0x5585d0ce9140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f9ed054c89adc0 server_cookie=cd3893a045e8ef3e in_seq=0 out_seq=0 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.584+0000 7fe192ffd640 1 -- 172.21.15.50:0/1178303316 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5585d0bd13f0 con 0x5585d08541c0 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.584+0000 7fe192ffd640 10 client.25157 handle_command_reply: tid=0 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.584+0000 7fe1c0fea740 1 client.25157 shutdown 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.584+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 msgr2=0x5585d0c782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.584+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 0x5585d0c782a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80e00a0 tx=0x7fe1a80cc040 comp rx=0 tx=0).stop 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 shutdown_connections 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5585d08c01b0 0x5585d0ce4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585d089f4d0 0x5585d0c782a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5585d08541c0 0x5585d0ce9140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 --2- 172.21.15.50:0/1178303316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585d08972f0 0x5585d0c77d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe192ffd640 0 client.25157 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 >> 172.21.15.50:0/1178303316 conn(0x5585d08b4300 msgr2=0x5585d0c669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 shutdown_connections 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 1 -- 172.21.15.50:0/1178303316 wait complete. 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1c0fea740 20 client.25157 trim_cache size 0 max 16384 2024-05-17T04:37:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:37:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:37:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:37:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 msgr2=0x7fe19c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 0x7fe19c0b4cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000c00 tx=0x7fe1a40046c0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 msgr2=0x7fe1b4199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4199d20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800e9f0 tx=0x7fe1a800eeb0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 shutdown_connections 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b4108e20 0x7fe1b41a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1b41062b0 0x7fe1b4199d20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe19c0b27e0 0x7fe19c0b4cf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.585+0000 7fe1ba132640 1 --2- 172.21.15.50:0/1130693077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1b406b230 0x7fe1b41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.586+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 >> 172.21.15.50:0/1130693077 conn(0x7fe1b40fd880 msgr2=0x7fe1b4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.586+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 shutdown_connections 2024-05-17T04:37:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:26.586+0000 7fe1ba132640 1 -- 172.21.15.50:0/1130693077 wait complete. 2024-05-17T04:37:27.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:27 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:37:27.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:27 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:37:27.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:27 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:37:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:28 smithi105 ceph-mon[34795]: pgmap v473: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:37:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:37:28.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:37:28.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:37:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:28 smithi110 ceph-mon[35497]: pgmap v473: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:37:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:37:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:37:28.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:37:28.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:28 smithi050 ceph-mon[31739]: pgmap v473: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:37:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:37:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:37:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:37:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:30 smithi105 ceph-mon[34795]: pgmap v474: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:30.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:30 smithi110 ceph-mon[35497]: pgmap v474: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:30.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:30 smithi050 ceph-mon[31739]: pgmap v474: 129 pgs: 129 active+clean; 776 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:32 smithi105 ceph-mon[34795]: pgmap v475: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 504 op/s 2024-05-17T04:37:32.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:32 smithi110 ceph-mon[35497]: pgmap v475: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 504 op/s 2024-05-17T04:37:32.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:32 smithi050 ceph-mon[31739]: pgmap v475: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 504 op/s 2024-05-17T04:37:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:34 smithi105 ceph-mon[34795]: pgmap v476: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 521 op/s 2024-05-17T04:37:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:34 smithi110 ceph-mon[35497]: pgmap v476: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 521 op/s 2024-05-17T04:37:34.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:34 smithi050 ceph-mon[31739]: pgmap v476: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 521 op/s 2024-05-17T04:37:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:36 smithi105 ceph-mon[34795]: pgmap v477: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-05-17T04:37:36.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:36 smithi110 ceph-mon[35497]: pgmap v477: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-05-17T04:37:36.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:36 smithi050 ceph-mon[31739]: pgmap v477: 129 pgs: 129 active+clean; 776 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-05-17T04:37:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:38 smithi105 ceph-mon[34795]: pgmap v478: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-17T04:37:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:38 smithi110 ceph-mon[35497]: pgmap v478: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-17T04:37:38.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:38 smithi050 ceph-mon[31739]: pgmap v478: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 471 op/s 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.690+0000 7f9733d9e640 1 -- 172.21.15.50:0/4165418073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c1031b0 msgr2=0x7f972c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.690+0000 7f9733d9e640 1 --2- 172.21.15.50:0/4165418073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c1031b0 0x7f972c1035c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f971c009900 tx=0x7f971c02a410 comp rx=0 tx=0).stop 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 -- 172.21.15.50:0/4165418073 shutdown_connections 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 --2- 172.21.15.50:0/4165418073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c103b00 0x7f972c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 --2- 172.21.15.50:0/4165418073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c1031b0 0x7f972c1035c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 --2- 172.21.15.50:0/4165418073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 0x7f972c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 -- 172.21.15.50:0/4165418073 >> 172.21.15.50:0/4165418073 conn(0x7f972c0fd8b0 msgr2=0x7f972c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 -- 172.21.15.50:0/4165418073 shutdown_connections 2024-05-17T04:37:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 -- 172.21.15.50:0/4165418073 wait complete. 2024-05-17T04:37:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 Processor -- start 2024-05-17T04:37:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.691+0000 7f9733d9e640 1 -- start start 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 0x7f972c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 0x7f972c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10da70 con 0x7f972c103b00 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10d8f0 con 0x7f972c1031b0 2024-05-17T04:37:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9733d9e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10dbf0 con 0x7f972c102090 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53162/0 (socket says 172.21.15.50:53162) 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 -- 172.21.15.50:0/1845468999 learned_addr learned my addr 172.21.15.50:0/1845468999 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9731339640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 0x7f972c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9731b3a640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 0x7f972c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 -- 172.21.15.50:0/1845468999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 msgr2=0x7f972c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 0x7f972c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 -- 172.21.15.50:0/1845468999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 msgr2=0x7f972c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 0x7f972c19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 -- 172.21.15.50:0/1845468999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c1a1360 con 0x7f972c103b00 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9731339640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 0x7f972c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f973233b640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f972400eb10 tx=0x7f972400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.692+0000 7f9722ffd640 1 -- 172.21.15.50:0/1845468999 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724004b20 con 0x7f972c103b00 2024-05-17T04:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.693+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c1a1650 con 0x7f972c103b00 2024-05-17T04:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.693+0000 7f9722ffd640 1 -- 172.21.15.50:0/1845468999 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9724004cc0 con 0x7f972c103b00 2024-05-17T04:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.693+0000 7f9722ffd640 1 -- 172.21.15.50:0/1845468999 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724010690 con 0x7f972c103b00 2024-05-17T04:37:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.693+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c1a1ba0 con 0x7f972c103b00 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.695+0000 7f9722ffd640 1 -- 172.21.15.50:0/1845468999 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9724005020 con 0x7f972c103b00 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 Processor -- start 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 -- start start 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71af1b0 0x5638a7552770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a7559c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638a74c4cc0 con 0x5638a71862f0 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638a74b4860 con 0x5638a718e4d0 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f973ac5a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5638a74b88a0 con 0x5638a71af1b0 2024-05-17T04:37:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53172/0 (socket says 172.21.15.50:53172) 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f9731b3a640 1 -- 172.21.15.50:0/1455796693 learned_addr learned my addr 172.21.15.50:0/1455796693 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f9731339640 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a7559c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.696+0000 7f9731339640 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71af1b0 0x5638a7552770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 -- 172.21.15.50:0/1455796693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71af1b0 msgr2=0x5638a7552770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71af1b0 0x5638a7552770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 -- 172.21.15.50:0/1455796693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 msgr2=0x5638a7559c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a7559c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9722ffd640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 0x7f97180b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 -- 172.21.15.50:0/1455796693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a747cd50 con 0x5638a71862f0 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f973233b640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 0x7f97180b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9731b3a640 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9728009ca0 tx=0x7f9728011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a25265cf4e149 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9722ffd640 1 -- 172.21.15.50:0/1845468999 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97240e2b50 con 0x7f972c103b00 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9717fff640 1 -- 172.21.15.50:0/1455796693 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972801e070 con 0x5638a71862f0 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9717fff640 1 -- 172.21.15.50:0/1455796693 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9728019ac0 con 0x5638a71862f0 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.697+0000 7f9717fff640 1 -- 172.21.15.50:0/1455796693 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972800e810 con 0x5638a71862f0 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 -- 172.21.15.50:0/1455796693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 msgr2=0x5638a7552cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973233b640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 0x7f97180b4cf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f97240049d0 tx=0x7f9724074600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9728009ca0 tx=0x7f9728011810 comp rx=0 tx=0).stop 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 -- 172.21.15.50:0/1455796693 shutdown_connections 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a7559c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71862f0 0x5638a7552cf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 --2- 172.21.15.50:0/1455796693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71af1b0 0x5638a7552770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 -- 172.21.15.50:0/1455796693 >> 172.21.15.50:0/1455796693 conn(0x5638a71a3300 msgr2=0x5638a754e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 -- 172.21.15.50:0/1455796693 shutdown_connections 2024-05-17T04:37:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.698+0000 7f973ac5a740 1 -- 172.21.15.50:0/1455796693 wait complete. 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 -- start start 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 Processor -- start 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a755c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638a73f0ab0 con 0x5638a71af1b0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638a74714d0 con 0x5638a718e4d0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5638a755a690 con 0x5638a71862f0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f973ac5a740 10 client.0 _ll_register_callbacks cb 0x5638a74af950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f9731339640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:32838/0 (socket says 172.21.15.50:32838) 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f9731b3a640 1 -- 172.21.15.50:0/3096370101 learned_addr learned my addr 172.21.15.50:0/3096370101 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.699+0000 7f9731339640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a755c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 -- 172.21.15.50:0/3096370101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 msgr2=0x5638a755c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a755c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 -- 172.21.15.50:0/3096370101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 msgr2=0x5638a75634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 -- 172.21.15.50:0/3096370101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a7472340 con 0x5638a718e4d0 2024-05-17T04:37:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f9731b3a640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f972801c040 tx=0x7f97280098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b073011adad1e4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f97157fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972801e050 con 0x5638a718e4d0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9728019ac0 con 0x5638a718e4d0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9728005590 con 0x5638a718e4d0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9728033050 con 0x5638a718e4d0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f973ac5a740 10 client.25169 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.700+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5638a721a790 con 0x5638a718e4d0 2024-05-17T04:37:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9728019ac0 con 0x5638a718e4d0 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f972800e8c0 con 0x5638a718e4d0 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 10 client.25169 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 10 client.25169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 20 client.25169 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a716dd80 0x5638a7569610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 4 client.25169 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a74c6c10 con 0x5638a716dd80 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f9731339640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a716dd80 0x5638a7569610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f9731339640 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a716dd80 0x5638a7569610 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d4eec0dbbcdf88 server_cookie=be2bed97734a048d in_seq=0 out_seq=0 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.701+0000 7f97157fa640 10 client.25169 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f97157fa640 1 -- 172.21.15.50:0/3096370101 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638a74c6c10 con 0x5638a716dd80 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f97157fa640 10 client.25169 handle_command_reply: tid=0 2024-05-17T04:37:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 client.25169 shutdown 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 msgr2=0x5638a755c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f972801c040 tx=0x7f97280098d0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 shutdown_connections 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a716dd80 0x5638a7569610 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a755c540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3096370101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a755c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f97157fa640 0 client.25169 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.702+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 >> 172.21.15.50:0/3096370101 conn(0x5638a71a3300 msgr2=0x5638a7551010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.703+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 shutdown_connections 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.703+0000 7f973ac5a740 1 -- 172.21.15.50:0/3096370101 wait complete. 2024-05-17T04:37:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.703+0000 7f973ac5a740 20 client.25169 trim_cache size 0 max 16384 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 Processor -- start 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 -- start start 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a7556810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1ae860 con 0x5638a71af1b0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1ac0d0 con 0x5638a718e4d0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f973ac5a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1a9f20 con 0x5638a71862f0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53186/0 (socket says 172.21.15.50:53186) 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731b3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:32848/0 (socket says 172.21.15.50:32848) 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 -- 172.21.15.50:0/3246856378 learned_addr learned my addr 172.21.15.50:0/3246856378 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a7556810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 -- 172.21.15.50:0/3246856378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 msgr2=0x5638a7556810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a7556810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 -- 172.21.15.50:0/3246856378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 msgr2=0x5638a75514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 -- 172.21.15.50:0/3246856378 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a7293620 con 0x5638a71af1b0 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731b3a640 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.719+0000 7f9731339640 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f971c0057c0 tx=0x7f971c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf49e31cbaa1247 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.720+0000 7f97177fe640 1 -- 172.21.15.50:0/3246856378 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c035040 con 0x5638a71af1b0 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.720+0000 7f97177fe640 1 -- 172.21.15.50:0/3246856378 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f971c0042e0 con 0x5638a71af1b0 2024-05-17T04:37:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.720+0000 7f97177fe640 1 -- 172.21.15.50:0/3246856378 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c050370 con 0x5638a71af1b0 2024-05-17T04:37:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- 172.21.15.50:0/3246856378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 msgr2=0x5638a7551a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f971c0057c0 tx=0x7f971c009260 comp rx=0 tx=0).stop 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- 172.21.15.50:0/3246856378 shutdown_connections 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a7551a20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a718e4d0 0x5638a75514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 --2- 172.21.15.50:0/3246856378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a71862f0 0x5638a7556810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- 172.21.15.50:0/3246856378 >> 172.21.15.50:0/3246856378 conn(0x5638a71a3300 msgr2=0x5638a7550dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- 172.21.15.50:0/3246856378 shutdown_connections 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- 172.21.15.50:0/3246856378 wait complete. 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 -- start start 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.721+0000 7f973ac5a740 1 Processor -- start 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a71862f0 0x5638a75615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a718e4d0 0x5638a7561b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1aa590 con 0x5638a71af1b0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1aba00 con 0x5638a71862f0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1aac60 con 0x5638a718e4d0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f973ac5a740 10 client.0 _ll_register_callbacks cb 0x7f972801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a71862f0 0x5638a75615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53192/0 (socket says 172.21.15.50:53192) 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 -- 172.21.15.50:0/2540129864 learned_addr learned my addr 172.21.15.50:0/2540129864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731b3a640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a718e4d0 0x5638a7561b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 -- 172.21.15.50:0/2540129864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a718e4d0 msgr2=0x5638a7561b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a718e4d0 0x5638a7561b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 -- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a71862f0 msgr2=0x5638a75615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a71862f0 0x5638a75615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 -- 172.21.15.50:0/2540129864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a7562380 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f9731339640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f971c005e50 tx=0x7f971c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1ed8511dc19a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f97157fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.722+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c035040 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f971c050110 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971c0042e0 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f971c003bf0 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 10 client.15393 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5638a75d0ed0 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f971c0042e0 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f971c03d710 con 0x5638a71af1b0 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 10 client.15393 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 20 client.15393 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a71431c0 0x5638a75d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 4 client.15393 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638a74c03f0 con 0x5638a71431c0 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.723+0000 7f9731b3a640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a71431c0 0x5638a75d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.724+0000 7f97157fa640 10 client.15393 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.724+0000 7f9731b3a640 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a71431c0 0x5638a75d3920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98483eeffd19691f server_cookie=f0bb3898e418704b in_seq=0 out_seq=0 2024-05-17T04:37:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f97157fa640 1 -- 172.21.15.50:0/2540129864 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5638a74c03f0 con 0x5638a71431c0 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f97157fa640 10 client.15393 handle_command_reply: tid=0 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 client.15393 shutdown 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 msgr2=0x5638a75cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f971c005e50 tx=0x7f971c005fd0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 shutdown_connections 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5638a71431c0 0x5638a75d3920 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638a71af1b0 0x5638a75cf970 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5638a718e4d0 0x5638a7561b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f97157fa640 0 client.15393 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 --2- 172.21.15.50:0/2540129864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638a71862f0 0x5638a75615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 >> 172.21.15.50:0/2540129864 conn(0x5638a71a3300 msgr2=0x5638a7551790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 shutdown_connections 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 1 -- 172.21.15.50:0/2540129864 wait complete. 2024-05-17T04:37:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.736+0000 7f973ac5a740 20 client.15393 trim_cache size 0 max 16384 2024-05-17T04:37:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:37:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:37:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 msgr2=0x7f97180b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 0x7f97180b4cf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f97240049d0 tx=0x7f9724074600 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 msgr2=0x7f972c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f972400eb10 tx=0x7f972400c2d0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 shutdown_connections 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97180b27e0 0x7f97180b4cf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f972c103b00 0x7f972c1a0b40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f972c1031b0 0x7f972c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 --2- 172.21.15.50:0/1845468999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f972c102090 0x7f972c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 >> 172.21.15.50:0/1845468999 conn(0x7f972c0fd8b0 msgr2=0x7f972c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 shutdown_connections 2024-05-17T04:37:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:39.737+0000 7f9733d9e640 1 -- 172.21.15.50:0/1845468999 wait complete. 2024-05-17T04:37:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:40 smithi105 ceph-mon[34795]: pgmap v479: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-05-17T04:37:40.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:40 smithi110 ceph-mon[35497]: pgmap v479: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-05-17T04:37:40.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:40 smithi050 ceph-mon[31739]: pgmap v479: 129 pgs: 129 active+clean; 776 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 560 op/s 2024-05-17T04:37:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:42 smithi105 ceph-mon[34795]: pgmap v480: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 559 op/s 2024-05-17T04:37:42.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:42 smithi110 ceph-mon[35497]: pgmap v480: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 559 op/s 2024-05-17T04:37:42.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:42 smithi050 ceph-mon[31739]: pgmap v480: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 559 op/s 2024-05-17T04:37:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:44 smithi105 ceph-mon[34795]: pgmap v481: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:44.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:44 smithi110 ceph-mon[35497]: pgmap v481: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:44.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:44 smithi050 ceph-mon[31739]: pgmap v481: 129 pgs: 129 active+clean; 776 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2024-05-17T04:37:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:46 smithi105 ceph-mon[34795]: pgmap v482: 129 pgs: 129 active+clean; 776 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:37:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:46 smithi110 ceph-mon[35497]: pgmap v482: 129 pgs: 129 active+clean; 776 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:37:46.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:46 smithi050 ceph-mon[31739]: pgmap v482: 129 pgs: 129 active+clean; 776 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:37:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:48 smithi105 ceph-mon[34795]: pgmap v483: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:48 smithi110 ceph-mon[35497]: pgmap v483: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:48.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:48 smithi050 ceph-mon[31739]: pgmap v483: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 463 op/s 2024-05-17T04:37:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:50 smithi105 ceph-mon[34795]: pgmap v484: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-17T04:37:50.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:50 smithi110 ceph-mon[35497]: pgmap v484: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-17T04:37:50.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:50 smithi050 ceph-mon[31739]: pgmap v484: 129 pgs: 129 active+clean; 776 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 464 op/s 2024-05-17T04:37:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:52 smithi105 ceph-mon[34795]: pgmap v485: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:37:52.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:52 smithi110 ceph-mon[35497]: pgmap v485: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:37:52.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:52 smithi050 ceph-mon[31739]: pgmap v485: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:37:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 -- 172.21.15.50:0/1169762123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58941031a0 msgr2=0x7f58941035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 --2- 172.21.15.50:0/1169762123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58941031a0 0x7f58941035b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f587c009900 tx=0x7f587c02a3f0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 -- 172.21.15.50:0/1169762123 shutdown_connections 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 --2- 172.21.15.50:0/1169762123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894103af0 0x7f589410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 --2- 172.21.15.50:0/1169762123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58941031a0 0x7f58941035b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 --2- 172.21.15.50:0/1169762123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894102080 0x7f5894102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 -- 172.21.15.50:0/1169762123 >> 172.21.15.50:0/1169762123 conn(0x7f58940fd880 msgr2=0x7f58940ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 -- 172.21.15.50:0/1169762123 shutdown_connections 2024-05-17T04:37:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.880+0000 7f589952b640 1 -- 172.21.15.50:0/1169762123 wait complete. 2024-05-17T04:37:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.881+0000 7f589952b640 1 Processor -- start 2024-05-17T04:37:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 -- start start 2024-05-17T04:37:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58941031a0 0x7f5894198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 0x7f589419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f589410dab0 con 0x7f5894102080 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f589410d930 con 0x7f5894103af0 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f589952b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f589410dc30 con 0x7f58941031a0 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49994/0 (socket says 172.21.15.50:49994) 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f5892ffd640 1 -- 172.21.15.50:0/611834989 learned_addr learned my addr 172.21.15.50:0/611834989 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f58937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 0x7f589419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.882+0000 7f588bfff640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58941031a0 0x7f5894198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 -- 172.21.15.50:0/611834989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58941031a0 msgr2=0x7f5894198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58941031a0 0x7f5894198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 -- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 msgr2=0x7f5894198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 -- 172.21.15.50:0/611834989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941a0270 con 0x7f5894103af0 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f5892ffd640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f58937fe640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 0x7f589419fc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f588c009fd0 tx=0x7f588c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.883+0000 7f5890ff9640 1 -- 172.21.15.50:0/611834989 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00ce20 con 0x7f5894103af0 2024-05-17T04:37:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.884+0000 7f5890ff9640 1 -- 172.21.15.50:0/611834989 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f588c005020 con 0x7f5894103af0 2024-05-17T04:37:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.884+0000 7f5890ff9640 1 -- 172.21.15.50:0/611834989 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c010690 con 0x7f5894103af0 2024-05-17T04:37:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.884+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589406a5c0 con 0x7f5894103af0 2024-05-17T04:37:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.884+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589406aa90 con 0x7f5894103af0 2024-05-17T04:37:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.887+0000 7f58a03df740 1 Processor -- start 2024-05-17T04:37:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.888+0000 7f5890ff9640 1 -- 172.21.15.50:0/611834989 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f588c010830 con 0x7f5894103af0 2024-05-17T04:37:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.888+0000 7f58a03df740 1 -- start start 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 0x55f72bbc6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f588bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 0x55f72bbc6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f588bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 0x55f72bbc6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49998/0 (socket says 172.21.15.50:49998) 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f588bfff640 1 -- 172.21.15.50:0/1611950635 learned_addr learned my addr 172.21.15.50:0/1611950635 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f72bb38cc0 con 0x55f72b8231b0 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f72bb28860 con 0x55f72b7fa2f0 2024-05-17T04:37:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f72bb2c8a0 con 0x55f72b8024d0 2024-05-17T04:37:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 -- 172.21.15.50:0/1611950635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 msgr2=0x55f72bbcdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 -- 172.21.15.50:0/1611950635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 msgr2=0x55f72bbc6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 0x55f72bbc6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.889+0000 7f5892ffd640 1 -- 172.21.15.50:0/1611950635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72baf0d50 con 0x55f72b7fa2f0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f5890ff9640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 0x7f58800b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f588bfff640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 0x7f58800b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f5892ffd640 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc6cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5884010450 tx=0x7f5884010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f357bd3e3d5b622 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f5890ff9640 1 -- 172.21.15.50:0/611834989 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f588c0e2c90 con 0x7f5894103af0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f58897fa640 1 -- 172.21.15.50:0/1611950635 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884011820 con 0x55f72b7fa2f0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f58897fa640 1 -- 172.21.15.50:0/1611950635 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5884011e50 con 0x55f72b7fa2f0 2024-05-17T04:37:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.890+0000 7f58897fa640 1 -- 172.21.15.50:0/1611950635 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588401cd40 con 0x55f72b7fa2f0 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.891+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 msgr2=0x55f72bbc6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.891+0000 7f58a03df740 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc6cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5884010450 tx=0x7f5884010910 comp rx=0 tx=0).stop 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.891+0000 7f588bfff640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 0x7f58800b4df0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f587c006010 tx=0x7f587c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 shutdown_connections 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc6cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 --2- 172.21.15.50:0/1611950635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8231b0 0x55f72bbc6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 >> 172.21.15.50:0/1611950635 conn(0x55f72b817300 msgr2=0x55f72bbc21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 shutdown_connections 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 -- 172.21.15.50:0/1611950635 wait complete. 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.892+0000 7f58a03df740 1 -- start start 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 Processor -- start 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 0x55f72bbcbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 0x55f72bbccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f72ba64ab0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f72bae54d0 con 0x55f72b8231b0 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f72bbce630 con 0x55f72b8024d0 2024-05-17T04:37:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58a03df740 10 client.0 _ll_register_callbacks cb 0x55f72bb23950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 0x55f72bbcbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 0x55f72bbcbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50012/0 (socket says 172.21.15.50:50012) 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f5892ffd640 1 -- 172.21.15.50:0/3851087422 learned_addr learned my addr 172.21.15.50:0/3851087422 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f58937fe640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 0x55f72bbccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.893+0000 7f588bfff640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 -- 172.21.15.50:0/3851087422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 msgr2=0x55f72bbcc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 -- 172.21.15.50:0/3851087422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 msgr2=0x55f72bbcbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 0x55f72bbcbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 -- 172.21.15.50:0/3851087422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72bae6340 con 0x55f72b8231b0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58937fe640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 0x55f72bbccff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a9e50 tx=0x7f588c0aa680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3303a3d79c11345 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f587a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0cc040 con 0x55f72b8231b0 2024-05-17T04:37:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f588c0051d0 con 0x55f72b8231b0 2024-05-17T04:37:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.894+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5884011e50 con 0x55f72b8231b0 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f588c0b2d20 con 0x55f72b8231b0 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 10 client.25187 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f72b88e790 con 0x55f72b8231b0 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5884011e50 con 0x55f72b8231b0 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f588c0a5830 con 0x55f72b8231b0 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 10 client.25187 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 10 client.25187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 20 client.25187 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7e1d80 0x55f72bbe6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 4 client.25187 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.895+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72bb3ac10 con 0x55f72b7e1d80 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.896+0000 7f5892ffd640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7e1d80 0x55f72bbe6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.896+0000 7f587a7fc640 10 client.25187 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.896+0000 7f5892ffd640 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7e1d80 0x55f72bbe6200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89277d10da6bf07f server_cookie=f7f0d789e9edbc5a in_seq=0 out_seq=0 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f587a7fc640 1 -- 172.21.15.50:0/3851087422 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f72bb3ac10 con 0x55f72b7e1d80 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f587a7fc640 10 client.25187 handle_command_reply: tid=0 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 client.25187 shutdown 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 msgr2=0x55f72bbccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 0x55f72bbccff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a9e50 tx=0x7f588c0aa680 comp rx=0 tx=0).stop 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 shutdown_connections 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b8231b0 0x55f72bbccff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8024d0 0x55f72bbcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7e1d80 0x55f72bbe6200 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 --2- 172.21.15.50:0/3851087422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b7fa2f0 0x55f72bbcbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 >> 172.21.15.50:0/3851087422 conn(0x55f72b817300 msgr2=0x55f72bbc5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f587a7fc640 0 client.25187 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 shutdown_connections 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.897+0000 7f58a03df740 1 -- 172.21.15.50:0/3851087422 wait complete. 2024-05-17T04:37:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.898+0000 7f58a03df740 20 client.25187 trim_cache size 0 max 16384 2024-05-17T04:37:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.923+0000 7f58a03df740 1 Processor -- start 2024-05-17T04:37:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.923+0000 7f58a03df740 1 -- start start 2024-05-17T04:37:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.923+0000 7f58a03df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58a03df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58a03df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbc4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58a03df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58941ac670 con 0x55f72b8024d0 2024-05-17T04:37:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58a03df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58941a9ee0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58a03df740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58941a7d30 con 0x55f72b8231b0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f588bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f588bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50020/0 (socket says 172.21.15.50:50020) 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f588bfff640 1 -- 172.21.15.50:0/2077626133 learned_addr learned my addr 172.21.15.50:0/2077626133 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbc4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 -- 172.21.15.50:0/2077626133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 msgr2=0x55f72bbc1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 -- 172.21.15.50:0/2077626133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 msgr2=0x55f72bbc1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 -- 172.21.15.50:0/2077626133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72b907620 con 0x55f72b8231b0 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f588bfff640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f5892ffd640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:37:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.924+0000 7f58937fe640 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbc4d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a7130 tx=0x7f588c0aa6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9322a3e241e2a971 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.925+0000 7f5888ff9640 1 -- 172.21.15.50:0/2077626133 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0cc070 con 0x55f72b8231b0 2024-05-17T04:37:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.925+0000 7f5888ff9640 1 -- 172.21.15.50:0/2077626133 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f588c0051d0 con 0x55f72b8231b0 2024-05-17T04:37:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.925+0000 7f5888ff9640 1 -- 172.21.15.50:0/2077626133 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0b2890 con 0x55f72b8231b0 2024-05-17T04:37:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- 172.21.15.50:0/2077626133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 msgr2=0x55f72bbc4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbc4d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a7130 tx=0x7f588c0aa6f0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- 172.21.15.50:0/2077626133 shutdown_connections 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbc4d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbc1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 --2- 172.21.15.50:0/2077626133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbc1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- 172.21.15.50:0/2077626133 >> 172.21.15.50:0/2077626133 conn(0x55f72b817300 msgr2=0x55f72bbca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- 172.21.15.50:0/2077626133 shutdown_connections 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- 172.21.15.50:0/2077626133 wait complete. 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.926+0000 7f58a03df740 1 -- start start 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 Processor -- start 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbdd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f58941a83a0 con 0x55f72b8024d0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f588bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbdd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44820/0 (socket says 172.21.15.50:44820) 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 -- 172.21.15.50:0/247122350 learned_addr learned my addr 172.21.15.50:0/247122350 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58941a9810 con 0x55f72b7fa2f0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58937fe640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58941a8a70 con 0x55f72b8231b0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f58a03df740 10 client.0 _ll_register_callbacks cb 0x7f588c0a5d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 -- 172.21.15.50:0/247122350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 msgr2=0x55f72bbddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 -- 172.21.15.50:0/247122350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 msgr2=0x55f72bbdd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbdd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.927+0000 7f5892ffd640 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72bc4c5f0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f5892ffd640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55f72bbcb160 tx=0x7f588400bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47383da45e0b98b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f587a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884030060 con 0x55f72b7fa2f0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5884005590 con 0x55f72b7fa2f0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f588c0051d0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.928+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5884037050 con 0x55f72b7fa2f0 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 10 client.25193 fetch_fsmap learned FSMap version 31 2024-05-17T04:37:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f72bc4d870 con 0x55f72b7fa2f0 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f588c0051d0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5884011ba0 con 0x55f72b7fa2f0 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 10 client.25193 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 10 client.25193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 20 client.25193 populate_metadata read hostname 'smithi050' 2024-05-17T04:37:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7b71c0 0x55f72bc4ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:37:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 4 client.25193 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:37:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f72bb343f0 con 0x55f72b7b71c0 2024-05-17T04:37:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.929+0000 7f588bfff640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7b71c0 0x55f72bc4ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:37:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.930+0000 7f587a7fc640 10 client.25193 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.930+0000 7f588bfff640 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7b71c0 0x55f72bc4ffe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6b2262c7937ba server_cookie=1ee80d73adf6a0e3 in_seq=0 out_seq=0 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.940+0000 7f587a7fc640 1 -- 172.21.15.50:0/247122350 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f72bb343f0 con 0x55f72b7b71c0 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.940+0000 7f587a7fc640 10 client.25193 handle_command_reply: tid=0 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.940+0000 7f58a03df740 1 client.25193 shutdown 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 msgr2=0x55f72bbe0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55f72bbcb160 tx=0x7f588400bda0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 shutdown_connections 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f72b8231b0 0x55f72bbddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f72b7b71c0 0x55f72bc4ffe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f587a7fc640 0 client.25193 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f72b8024d0 0x55f72bbdd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 --2- 172.21.15.50:0/247122350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f72b7fa2f0 0x55f72bbe0920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 >> 172.21.15.50:0/247122350 conn(0x55f72b817300 msgr2=0x55f72bbc4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 shutdown_connections 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 1 -- 172.21.15.50:0/247122350 wait complete. 2024-05-17T04:37:52.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.941+0000 7f58a03df740 20 client.25193 trim_cache size 0 max 16384 2024-05-17T04:37:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:37:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:37:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:37:52.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 msgr2=0x7f58800b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 0x7f58800b4df0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f587c006010 tx=0x7f587c0095c0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 msgr2=0x7f589419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 0x7f589419fc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f588c009fd0 tx=0x7f588c00ef10 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 shutdown_connections 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5894103af0 0x7f589419fc00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58941031a0 0x7f5894198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f58800b28e0 0x7f58800b4df0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 --2- 172.21.15.50:0/611834989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5894102080 0x7f5894198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 >> 172.21.15.50:0/611834989 conn(0x7f58940fd880 msgr2=0x7f5894106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 shutdown_connections 2024-05-17T04:37:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:37:52.942+0000 7f589952b640 1 -- 172.21.15.50:0/611834989 wait complete. 2024-05-17T04:37:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:54 smithi105 ceph-mon[34795]: pgmap v486: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 508 op/s 2024-05-17T04:37:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:54 smithi110 ceph-mon[35497]: pgmap v486: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 508 op/s 2024-05-17T04:37:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:54 smithi050 ceph-mon[31739]: pgmap v486: 129 pgs: 129 active+clean; 776 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 508 op/s 2024-05-17T04:37:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:56 smithi105 ceph-mon[34795]: pgmap v487: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 581 op/s 2024-05-17T04:37:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:56 smithi110 ceph-mon[35497]: pgmap v487: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 581 op/s 2024-05-17T04:37:56.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:56 smithi050 ceph-mon[31739]: pgmap v487: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 581 op/s 2024-05-17T04:37:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:37:58 smithi105 ceph-mon[34795]: pgmap v488: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:37:58.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:37:58 smithi110 ceph-mon[35497]: pgmap v488: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:37:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:37:58 smithi050 ceph-mon[31739]: pgmap v488: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:38:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:00 smithi105 ceph-mon[34795]: pgmap v489: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 466 op/s 2024-05-17T04:38:00.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:00 smithi110 ceph-mon[35497]: pgmap v489: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 466 op/s 2024-05-17T04:38:00.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:00 smithi050 ceph-mon[31739]: pgmap v489: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 466 op/s 2024-05-17T04:38:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:02 smithi105 ceph-mon[34795]: pgmap v490: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 628 op/s 2024-05-17T04:38:02.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:02 smithi110 ceph-mon[35497]: pgmap v490: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 628 op/s 2024-05-17T04:38:02.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:02 smithi050 ceph-mon[31739]: pgmap v490: 129 pgs: 129 active+clean; 776 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 628 op/s 2024-05-17T04:38:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:04 smithi105 ceph-mon[34795]: pgmap v491: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-05-17T04:38:04.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:04 smithi110 ceph-mon[35497]: pgmap v491: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-05-17T04:38:04.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:04 smithi050 ceph-mon[31739]: pgmap v491: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 609 op/s 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 -- 172.21.15.50:0/187147200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 msgr2=0x7f69801089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 --2- 172.21.15.50:0/187147200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f69801089f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6974009900 tx=0x7f697402a410 comp rx=0 tx=0).stop 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 -- 172.21.15.50:0/187147200 shutdown_connections 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 --2- 172.21.15.50:0/187147200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980108f30 0x7f698010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 --2- 172.21.15.50:0/187147200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f69801089f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 --2- 172.21.15.50:0/187147200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980103c70 0x7f6980106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.072+0000 7f6985903640 1 -- 172.21.15.50:0/187147200 >> 172.21.15.50:0/187147200 conn(0x7f69800fd880 msgr2=0x7f69800ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.073+0000 7f6985903640 1 -- 172.21.15.50:0/187147200 shutdown_connections 2024-05-17T04:38:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.073+0000 7f6985903640 1 -- 172.21.15.50:0/187147200 wait complete. 2024-05-17T04:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.073+0000 7f6985903640 1 Processor -- start 2024-05-17T04:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 -- start start 2024-05-17T04:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f6980197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 0x7f69801945e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f698010eba0 con 0x7f69801065d0 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f698010ea20 con 0x7f6980103c70 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f6980197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36246/0 (socket says 172.21.15.50:36246) 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697effd640 1 -- 172.21.15.50:0/290170147 learned_addr learned my addr 172.21.15.50:0/290170147 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f6980197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43644/0 (socket says 172.21.15.50:43644) 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f698010ed20 con 0x7f6980108f30 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.074+0000 7f697f7fe640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 0x7f69801945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 -- 172.21.15.50:0/290170147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 msgr2=0x7f69801945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 0x7f69801945e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 -- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 msgr2=0x7f6980197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f6980197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 -- 172.21.15.50:0/290170147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6980194e00 con 0x7f6980103c70 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697f7fe640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 0x7f69801945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f697effd640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9c0 tx=0x7f697000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f6984901640 1 -- 172.21.15.50:0/290170147 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cd80 con 0x7f6980103c70 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f6984901640 1 -- 172.21.15.50:0/290170147 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69700035f0 con 0x7f6980103c70 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f6984901640 1 -- 172.21.15.50:0/290170147 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69700044c0 con 0x7f6980103c70 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801950f0 con 0x7f6980103c70 2024-05-17T04:38:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.075+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a8150 con 0x7f6980103c70 2024-05-17T04:38:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 Processor -- start 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 -- start start 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66602f0 0x55a0b6a2ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 0x55a0b6a33c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a0b699ecc0 con 0x55a0b66684d0 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0b698e860 con 0x55a0b66602f0 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.078+0000 7f698c7bf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a0b69928a0 con 0x55a0b66891b0 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:38234/0 (socket says 172.21.15.50:38234) 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 -- 172.21.15.50:0/434597911 learned_addr learned my addr 172.21.15.50:0/434597911 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697e7fc640 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 0x55a0b6a33c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 -- 172.21.15.50:0/434597911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66602f0 msgr2=0x55a0b6a2ccf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66602f0 0x55a0b6a2ccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f6984901640 1 -- 172.21.15.50:0/290170147 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6970010750 con 0x7f6980103c70 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 -- 172.21.15.50:0/434597911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 msgr2=0x55a0b6a33c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 0x55a0b6a33c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 -- 172.21.15.50:0/434597911 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b6956d50 con 0x55a0b66891b0 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697e7fc640 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 0x55a0b6a33c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f697f7fe640 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fd0 tx=0x7f69780108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36126b438e3746c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.079+0000 7f6965ffb640 1 -- 172.21.15.50:0/434597911 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978011760 con 0x55a0b66891b0 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.080+0000 7f6965ffb640 1 -- 172.21.15.50:0/434597911 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6978011d90 con 0x55a0b66891b0 2024-05-17T04:38:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.080+0000 7f6965ffb640 1 -- 172.21.15.50:0/434597911 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800e3c0 con 0x55a0b66891b0 2024-05-17T04:38:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.080+0000 7f6984901640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 0x7f69680b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.080+0000 7f6984901640 1 -- 172.21.15.50:0/290170147 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f69700e2a10 con 0x7f6980103c70 2024-05-17T04:38:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.080+0000 7f697effd640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 0x7f69680b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- 172.21.15.50:0/434597911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 msgr2=0x55a0b6a2c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fd0 tx=0x7f69780108e0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f697effd640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 0x7f69680b4d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f69700a9bd0 tx=0x7f69700096e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- 172.21.15.50:0/434597911 shutdown_connections 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66684d0 0x55a0b6a33c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66602f0 0x55a0b6a2ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 --2- 172.21.15.50:0/434597911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a2c770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- 172.21.15.50:0/434597911 >> 172.21.15.50:0/434597911 conn(0x55a0b667d300 msgr2=0x55a0b6a281a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- 172.21.15.50:0/434597911 shutdown_connections 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- 172.21.15.50:0/434597911 wait complete. 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.081+0000 7f698c7bf740 1 -- start start 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 Processor -- start 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43662/0 (socket says 172.21.15.50:43662) 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f697f7fe640 1 -- 172.21.15.50:0/1693112122 learned_addr learned my addr 172.21.15.50:0/1693112122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a0b68caab0 con 0x55a0b66602f0 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0b694b4d0 con 0x55a0b66684d0 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6a34730 con 0x55a0b66891b0 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.082+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 -- 172.21.15.50:0/1693112122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 msgr2=0x55a0b6a3d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a3d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 -- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 msgr2=0x55a0b6a36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a36640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b694c340 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f698c7bf740 10 client.0 _ll_register_callbacks cb 0x55a0b6989950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6978010fd0 tx=0x7f6978009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680777b2646aa40d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f695effd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6978011d90 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f695effd640 1 -- 172.21.15.50:0/1693112122 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978011770 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.083+0000 7f695effd640 1 -- 172.21.15.50:0/1693112122 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6978011910 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f695effd640 1 -- 172.21.15.50:0/1693112122 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6978011b10 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 10 client.15423 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a0b66f4790 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f695effd640 1 -- 172.21.15.50:0/1693112122 <== mon.0 v2:172.21.15.50:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6978029020 con 0x55a0b66602f0 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 10 client.15423 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 20 client.15423 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b6647d80 0x55a0b6a43380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 4 client.15423 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b69a0c10 con 0x55a0b6647d80 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.084+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b6647d80 0x55a0b6a43380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.085+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b6647d80 0x55a0b6a43380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e4b6263f5625ea server_cookie=e938e4ec079ab195 in_seq=0 out_seq=0 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.085+0000 7f695effd640 10 client.15423 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f695effd640 1 -- 172.21.15.50:0/1693112122 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0b69a0c10 con 0x55a0b6647d80 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f695effd640 10 client.15423 handle_command_reply: tid=0 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 client.15423 shutdown 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 msgr2=0x55a0b6a36100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6978010fd0 tx=0x7f6978009c10 comp rx=0 tx=0).stop 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 shutdown_connections 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66891b0 0x55a0b6a3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a36640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b6647d80 0x55a0b6a43380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1693112122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a36100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 >> 172.21.15.50:0/1693112122 conn(0x55a0b667d300 msgr2=0x55a0b6a271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f695effd640 0 client.15423 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 shutdown_connections 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 1 -- 172.21.15.50:0/1693112122 wait complete. 2024-05-17T04:38:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.086+0000 7f698c7bf740 20 client.15423 trim_cache size 0 max 16384 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 Processor -- start 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 -- start start 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 0x55a0b6a28200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66684d0 0x55a0b6a2ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b4f60 con 0x55a0b66602f0 2024-05-17T04:38:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b27d0 con 0x55a0b667d300 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f698c7bf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b0620 con 0x55a0b66684d0 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 0x55a0b6a28200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43676/0 (socket says 172.21.15.50:43676) 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.109+0000 7f697e7fc640 1 -- 172.21.15.50:0/1884743104 learned_addr learned my addr 172.21.15.50:0/1884743104 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66684d0 0x55a0b6a2ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 -- 172.21.15.50:0/1884743104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66684d0 msgr2=0x55a0b6a2ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66684d0 0x55a0b6a2ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 -- 172.21.15.50:0/1884743104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 msgr2=0x55a0b6a3d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 -- 172.21.15.50:0/1884743104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b676d620 con 0x55a0b667d300 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 0x55a0b6a28200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f69780092f0 tx=0x7f6978025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e68385efefb4614 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.110+0000 7f69657fa640 1 -- 172.21.15.50:0/1884743104 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697803d040 con 0x55a0b667d300 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.111+0000 7f69657fa640 1 -- 172.21.15.50:0/1884743104 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6978011d90 con 0x55a0b667d300 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.111+0000 7f69657fa640 1 -- 172.21.15.50:0/1884743104 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780057a0 con 0x55a0b667d300 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- 172.21.15.50:0/1884743104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 msgr2=0x55a0b6a28200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 0x55a0b6a28200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f69780092f0 tx=0x7f6978025000 comp rx=0 tx=0).stop 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- 172.21.15.50:0/1884743104 shutdown_connections 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b66684d0 0x55a0b6a2ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a3d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1884743104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b667d300 0x55a0b6a28200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- 172.21.15.50:0/1884743104 >> 172.21.15.50:0/1884743104 conn(0x55a0b6a35a30 msgr2=0x55a0b6a27af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- 172.21.15.50:0/1884743104 shutdown_connections 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- 172.21.15.50:0/1884743104 wait complete. 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 -- start start 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.112+0000 7f698c7bf740 1 Processor -- start 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a38260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b667d300 0x55a0b6aa96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a38260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36278/0 (socket says 172.21.15.50:36278) 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a38260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43692/0 (socket says 172.21.15.50:43692) 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f697e7fc640 1 -- 172.21.15.50:0/1137989888 learned_addr learned my addr 172.21.15.50:0/1137989888 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b0c90 con 0x55a0b66602f0 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b2100 con 0x55a0b66684d0 2024-05-17T04:38:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.113+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b1360 con 0x55a0b667d300 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f698c7bf740 10 client.0 _ll_register_callbacks cb 0x7f6978009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b667d300 0x55a0b6aa96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 -- 172.21.15.50:0/1137989888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b667d300 msgr2=0x55a0b6aa96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b667d300 0x55a0b6aa96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 -- 172.21.15.50:0/1137989888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 msgr2=0x55a0b6a38260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a38260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b6a3d7a0 con 0x55a0b66684d0 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6978011d90 con 0x55a0b66684d0 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f697e7fc640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69740057c0 tx=0x7f69740035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e810e3c4b973247a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f695effd640 10 client.25208 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f695effd640 1 -- 172.21.15.50:0/1137989888 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974035040 con 0x55a0b66684d0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.114+0000 7f695effd640 1 -- 172.21.15.50:0/1137989888 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6974003b70 con 0x55a0b66684d0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f695effd640 1 -- 172.21.15.50:0/1137989888 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6974003d70 con 0x55a0b66684d0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 10 client.25208 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a0b6aaab80 con 0x55a0b66684d0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f695effd640 1 -- 172.21.15.50:0/1137989888 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6974046380 con 0x55a0b66684d0 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 10 client.25208 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 10 client.25208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 20 client.25208 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b66891b0 0x55a0b6aad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.115+0000 7f698c7bf740 4 client.25208 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:38:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.116+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0b699a3f0 con 0x55a0b66891b0 2024-05-17T04:38:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.116+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b66891b0 0x55a0b6aad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.116+0000 7f697f7fe640 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b66891b0 0x55a0b6aad540 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ed5a90e9f62533 server_cookie=13e828b28def712b in_seq=0 out_seq=0 2024-05-17T04:38:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.116+0000 7f695effd640 10 client.25208 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.125+0000 7f695effd640 1 -- 172.21.15.50:0/1137989888 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0b699a3f0 con 0x55a0b66891b0 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.125+0000 7f695effd640 10 client.25208 handle_command_reply: tid=0 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.125+0000 7f698c7bf740 1 client.25208 shutdown 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.125+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 msgr2=0x55a0b6a387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.125+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69740057c0 tx=0x7f69740035f0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 shutdown_connections 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a0b667d300 0x55a0b6aa96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b66684d0 0x55a0b6a387a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a0b66891b0 0x55a0b6aad540 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 --2- 172.21.15.50:0/1137989888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0b66602f0 0x55a0b6a38260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 >> 172.21.15.50:0/1137989888 conn(0x55a0b6a35a30 msgr2=0x55a0b6a267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f695effd640 0 client.25208 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 shutdown_connections 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 1 -- 172.21.15.50:0/1137989888 wait complete. 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.126+0000 7f698c7bf740 20 client.25208 trim_cache size 0 max 16384 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:38:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:38:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 msgr2=0x7f69680b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 0x7f69680b4d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f69700a9bd0 tx=0x7f69700096e0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 msgr2=0x7f6980197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9c0 tx=0x7f697000ee80 comp rx=0 tx=0).stop 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 shutdown_connections 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6980108f30 0x7f69801945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f69680b2860 0x7f69680b4d70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f69801065d0 0x7f6980197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 --2- 172.21.15.50:0/290170147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6980103c70 0x7f6980197a90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 >> 172.21.15.50:0/290170147 conn(0x7f69800fd880 msgr2=0x7f6980101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 shutdown_connections 2024-05-17T04:38:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:06.127+0000 7f6985903640 1 -- 172.21.15.50:0/290170147 wait complete. 2024-05-17T04:38:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:06 smithi105 ceph-mon[34795]: pgmap v492: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-05-17T04:38:06.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:06 smithi110 ceph-mon[35497]: pgmap v492: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-05-17T04:38:06.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:06 smithi050 ceph-mon[31739]: pgmap v492: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-05-17T04:38:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:08 smithi105 ceph-mon[34795]: pgmap v493: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-05-17T04:38:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:08 smithi110 ceph-mon[35497]: pgmap v493: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-05-17T04:38:08.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:08 smithi050 ceph-mon[31739]: pgmap v493: 129 pgs: 129 active+clean; 776 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 456 op/s 2024-05-17T04:38:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:10 smithi105 ceph-mon[34795]: pgmap v494: 129 pgs: 129 active+clean; 776 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-05-17T04:38:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:10 smithi110 ceph-mon[35497]: pgmap v494: 129 pgs: 129 active+clean; 776 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-05-17T04:38:10.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:10 smithi050 ceph-mon[31739]: pgmap v494: 129 pgs: 129 active+clean; 776 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-05-17T04:38:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:12 smithi105 ceph-mon[34795]: pgmap v495: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-05-17T04:38:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:12 smithi110 ceph-mon[35497]: pgmap v495: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-05-17T04:38:12.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:12 smithi050 ceph-mon[31739]: pgmap v495: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-05-17T04:38:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:14 smithi105 ceph-mon[34795]: pgmap v496: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-17T04:38:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:14 smithi110 ceph-mon[35497]: pgmap v496: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-17T04:38:14.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:14 smithi050 ceph-mon[31739]: pgmap v496: 129 pgs: 129 active+clean; 776 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2024-05-17T04:38:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:16 smithi105 ceph-mon[34795]: pgmap v497: 129 pgs: 129 active+clean; 777 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:38:16.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:16 smithi110 ceph-mon[35497]: pgmap v497: 129 pgs: 129 active+clean; 777 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:38:16.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:16 smithi050 ceph-mon[31739]: pgmap v497: 129 pgs: 129 active+clean; 777 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 419 op/s 2024-05-17T04:38:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:18 smithi105 ceph-mon[34795]: pgmap v498: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 498 op/s 2024-05-17T04:38:18.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:18 smithi110 ceph-mon[35497]: pgmap v498: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 498 op/s 2024-05-17T04:38:18.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:18 smithi050 ceph-mon[31739]: pgmap v498: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 498 op/s 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 -- 172.21.15.50:0/402581218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c103af0 msgr2=0x7f802c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 --2- 172.21.15.50:0/402581218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c103af0 0x7f802c10a3f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8024009ca0 tx=0x7f802402a8c0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 -- 172.21.15.50:0/402581218 shutdown_connections 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 --2- 172.21.15.50:0/402581218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c103af0 0x7f802c10a3f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 --2- 172.21.15.50:0/402581218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 --2- 172.21.15.50:0/402581218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c102080 0x7f802c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 -- 172.21.15.50:0/402581218 >> 172.21.15.50:0/402581218 conn(0x7f802c0fd880 msgr2=0x7f802c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.259+0000 7f8030c4e640 1 -- 172.21.15.50:0/402581218 shutdown_connections 2024-05-17T04:38:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.260+0000 7f8030c4e640 1 -- 172.21.15.50:0/402581218 wait complete. 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.260+0000 7f8030c4e640 1 Processor -- start 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 -- start start 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c103af0 0x7f802c1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110db0 con 0x7f802c102080 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110c30 con 0x7f802c103af0 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8030c4e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110f30 con 0x7f802c1031a0 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f802a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f802a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39058/0 (socket says 172.21.15.50:39058) 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f802a59c640 1 -- 172.21.15.50:0/2487929177 learned_addr learned my addr 172.21.15.50:0/2487929177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f802ad9d640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c103af0 0x7f802c1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 -- 172.21.15.50:0/2487929177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c103af0 msgr2=0x7f802c1a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c103af0 0x7f802c1a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 -- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 msgr2=0x7f802c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f8029d9b640 1 -- 172.21.15.50:0/2487929177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c1a3500 con 0x7f802c1031a0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.261+0000 7f802a59c640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c19bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f801b7fe640 1 -- 172.21.15.50:0/2487929177 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000ccc0 con 0x7f802c1031a0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c1a37f0 con 0x7f802c1031a0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f801b7fe640 1 -- 172.21.15.50:0/2487929177 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200108c0 con 0x7f802c1031a0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c1a3d90 con 0x7f802c1031a0 2024-05-17T04:38:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.262+0000 7f801b7fe640 1 -- 172.21.15.50:0/2487929177 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200187c0 con 0x7f802c1031a0 2024-05-17T04:38:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.265+0000 7f801b7fe640 1 -- 172.21.15.50:0/2487929177 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8020018960 con 0x7f802c1031a0 2024-05-17T04:38:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.265+0000 7f8037beb740 1 Processor -- start 2024-05-17T04:38:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 -- start start 2024-05-17T04:38:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dbff3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dbffac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561dbf65cc0 con 0x5561dbc501b0 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5561dbf55860 con 0x5561dbc2f4d0 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.266+0000 7f8037beb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5561dbf598a0 con 0x5561dbc272f0 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:47728/0 (socket says 172.21.15.50:47728) 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 -- 172.21.15.50:0/3004120134 learned_addr learned my addr 172.21.15.50:0/3004120134 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f8029d9b640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dbffac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802ad9d640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dbff3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 -- 172.21.15.50:0/3004120134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 msgr2=0x5561dbffac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dbffac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 -- 172.21.15.50:0/3004120134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 msgr2=0x5561dbff3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dbff3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 -- 172.21.15.50:0/3004120134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dbf1dd50 con 0x5561dbc272f0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f801b7fe640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 0x7f80140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802ad9d640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dbff3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802a59c640 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f801c0093f0 tx=0x7f801c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720bf6ef16d36b50 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f801b7fe640 1 -- 172.21.15.50:0/2487929177 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f80200e28c0 con 0x7f802c1031a0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.267+0000 7f802ad9d640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 0x7f80140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.268+0000 7f8018ff9640 1 -- 172.21.15.50:0/3004120134 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c01b020 con 0x5561dbc272f0 2024-05-17T04:38:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.268+0000 7f8018ff9640 1 -- 172.21.15.50:0/3004120134 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f801c017a90 con 0x5561dbc272f0 2024-05-17T04:38:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.268+0000 7f8018ff9640 1 -- 172.21.15.50:0/3004120134 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c016b20 con 0x5561dbc272f0 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 -- 172.21.15.50:0/3004120134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 msgr2=0x5561dbff3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f801c0093f0 tx=0x7f801c005f90 comp rx=0 tx=0).stop 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f802ad9d640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 0x7f80140b4cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5561dbff4960 tx=0x7f8024031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 -- 172.21.15.50:0/3004120134 shutdown_connections 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dbffac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc272f0 0x5561dbff3cf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 --2- 172.21.15.50:0/3004120134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dbff3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 -- 172.21.15.50:0/3004120134 >> 172.21.15.50:0/3004120134 conn(0x5561dbc44300 msgr2=0x5561dbfef1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 -- 172.21.15.50:0/3004120134 shutdown_connections 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.269+0000 7f8037beb740 1 -- 172.21.15.50:0/3004120134 wait complete. 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 -- start start 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 Processor -- start 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc0045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561dbe91ab0 con 0x5561dbc2f4d0 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5561dbf124d0 con 0x5561dbc272f0 2024-05-17T04:38:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5561dbffb6e0 con 0x5561dbc501b0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8037beb740 10 client.0 _ll_register_callbacks cb 0x5561dbf50950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f802a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8029d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8029d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39076/0 (socket says 172.21.15.50:39076) 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f802a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42874/0 (socket says 172.21.15.50:42874) 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.270+0000 7f8029d9b640 1 -- 172.21.15.50:0/3166290998 learned_addr learned my addr 172.21.15.50:0/3166290998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802ad9d640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc0045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 -- 172.21.15.50:0/3166290998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 msgr2=0x5561dc0045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc0045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 -- 172.21.15.50:0/3166290998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 msgr2=0x5561dbffd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 -- 172.21.15.50:0/3166290998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dbf13340 con 0x5561dbc272f0 2024-05-17T04:38:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f8029d9b640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f802a59c640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f801c00f010 tx=0x7f801c017630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96444d2a633d7377 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f8011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.271+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c01b070 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f801c005240 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801c017a90 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f801c005460 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 10 client.25220 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561dbcbb790 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f801c017a90 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f801c028070 con 0x5561dbc272f0 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 10 client.25220 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 20 client.25220 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbc0ed80 0x5561dc00a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 4 client.25220 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.272+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561dbf67c10 con 0x5561dbc0ed80 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.273+0000 7f8029d9b640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbc0ed80 0x5561dc00a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.273+0000 7f8029d9b640 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbc0ed80 0x5561dc00a6b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac77fde741c8e2f server_cookie=62227309fed50bcc in_seq=0 out_seq=0 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.273+0000 7f8011ffb640 10 client.25220 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8011ffb640 1 -- 172.21.15.50:0/3166290998 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561dbf67c10 con 0x5561dbc0ed80 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8011ffb640 10 client.25220 handle_command_reply: tid=0 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 client.25220 shutdown 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 msgr2=0x5561dbffd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f801c00f010 tx=0x7f801c017630 comp rx=0 tx=0).stop 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 shutdown_connections 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc0045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.274+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbc0ed80 0x5561dc00a6b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc2f4d0 0x5561dbffd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 1 --2- 172.21.15.50:0/3166290998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbffd110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 >> 172.21.15.50:0/3166290998 conn(0x5561dbc44300 msgr2=0x5561dbff2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8011ffb640 0 client.25220 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 shutdown_connections 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 1 -- 172.21.15.50:0/3166290998 wait complete. 2024-05-17T04:38:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.275+0000 7f8037beb740 20 client.25220 trim_cache size 0 max 16384 2024-05-17T04:38:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 Processor -- start 2024-05-17T04:38:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 -- start start 2024-05-17T04:38:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbff69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dc004a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1b0b90 con 0x5561dbc501b0 2024-05-17T04:38:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ae400 con 0x5561dbc272f0 2024-05-17T04:38:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.295+0000 7f8037beb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ac250 con 0x5561dbc2f4d0 2024-05-17T04:38:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f802ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dc004a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f802a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbff69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:47754/0 (socket says 172.21.15.50:47754) 2024-05-17T04:38:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 -- 172.21.15.50:0/1373872939 learned_addr learned my addr 172.21.15.50:0/1373872939 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 -- 172.21.15.50:0/1373872939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 msgr2=0x5561dbff69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbff69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 -- 172.21.15.50:0/1373872939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 msgr2=0x5561dc004a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dc004a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 -- 172.21.15.50:0/1373872939 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dbd34620 con 0x5561dbc2f4d0 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f802ad9d640 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dc004a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8029d9b640 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80200a9900 tx=0x7f80200cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7976932ac9975044 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8013fff640 1 -- 172.21.15.50:0/1373872939 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020091040 con 0x5561dbc2f4d0 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8013fff640 1 -- 172.21.15.50:0/1373872939 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200b22f0 con 0x5561dbc2f4d0 2024-05-17T04:38:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.296+0000 7f8013fff640 1 -- 172.21.15.50:0/1373872939 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200b29b0 con 0x5561dbc2f4d0 2024-05-17T04:38:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.297+0000 7f8037beb740 1 -- 172.21.15.50:0/1373872939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 msgr2=0x5561dc0044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.297+0000 7f8037beb740 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80200a9900 tx=0x7f80200cc040 comp rx=0 tx=0).stop 2024-05-17T04:38:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 -- 172.21.15.50:0/1373872939 shutdown_connections 2024-05-17T04:38:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc501b0 0x5561dc004a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc2f4d0 0x5561dc0044d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 --2- 172.21.15.50:0/1373872939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc272f0 0x5561dbff69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 -- 172.21.15.50:0/1373872939 >> 172.21.15.50:0/1373872939 conn(0x5561dbc44300 msgr2=0x5561dbfee1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 -- 172.21.15.50:0/1373872939 shutdown_connections 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 -- 172.21.15.50:0/1373872939 wait complete. 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 -- start start 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 Processor -- start 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.298+0000 7f8037beb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dc0023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ac8c0 con 0x5561dbc272f0 2024-05-17T04:38:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1add30 con 0x5561dbc2f4d0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1acf90 con 0x5561dbc501b0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39090/0 (socket says 172.21.15.50:39090) 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 -- 172.21.15.50:0/2088485639 learned_addr learned my addr 172.21.15.50:0/2088485639 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802ad9d640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8037beb740 10 client.0 _ll_register_callbacks cb 0x7f80200e8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dc0023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 -- 172.21.15.50:0/2088485639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 msgr2=0x5561dc070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 -- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 msgr2=0x5561dc0023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dc0023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.299+0000 7f802a59c640 1 -- 172.21.15.50:0/2088485639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dc071000 con 0x5561dbc272f0 2024-05-17T04:38:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f802a59c640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5561dbfeab90 tx=0x7f801c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac906065c33de2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c01b040 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f801c034020 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561dbf613f0 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f801c005c50 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8037beb740 10 client.15447 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.300+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561dc072280 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5561dbf613f0 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f801c028050 con 0x5561dbc272f0 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 10 client.15447 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 20 client.15447 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbbe41c0 0x5561dc074d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 4 client.15447 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561dbedb750 con 0x5561dbbe41c0 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.301+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbbe41c0 0x5561dc074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.302+0000 7f8011ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.302+0000 7f8029d9b640 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbbe41c0 0x5561dc074d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b30d7b9daa68c8 server_cookie=cc9613a0ef7071e2 in_seq=0 out_seq=0 2024-05-17T04:38:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8011ffb640 1 -- 172.21.15.50:0/2088485639 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561dbedb750 con 0x5561dbbe41c0 2024-05-17T04:38:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8011ffb640 10 client.15447 handle_command_reply: tid=0 2024-05-17T04:38:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 client.15447 shutdown 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 msgr2=0x5561dc001e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5561dbfeab90 tx=0x7f801c005fb0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 shutdown_connections 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561dbc501b0 0x5561dc070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561dbc2f4d0 0x5561dc0023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.311+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5561dbbe41c0 0x5561dc074d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8037beb740 1 --2- 172.21.15.50:0/2088485639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561dbc272f0 0x5561dc001e70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8011ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 >> 172.21.15.50:0/2088485639 conn(0x5561dbc44300 msgr2=0x5561dbfedf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 shutdown_connections 2024-05-17T04:38:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8037beb740 1 -- 172.21.15.50:0/2088485639 wait complete. 2024-05-17T04:38:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.312+0000 7f8037beb740 20 client.15447 trim_cache size 0 max 16384 2024-05-17T04:38:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:38:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:38:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 msgr2=0x7f80140b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 0x7f80140b4cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5561dbff4960 tx=0x7f8024031000 comp rx=0 tx=0).stop 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 msgr2=0x7f802c19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c19bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 shutdown_connections 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f802c103af0 0x7f802c1a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f802c1031a0 0x7f802c19bf00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f80140b27e0 0x7f80140b4cf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 --2- 172.21.15.50:0/2487929177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c102080 0x7f802c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 >> 172.21.15.50:0/2487929177 conn(0x7f802c0fd880 msgr2=0x7f802c106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 shutdown_connections 2024-05-17T04:38:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:19.313+0000 7f8030c4e640 1 -- 172.21.15.50:0/2487929177 wait complete. 2024-05-17T04:38:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:20 smithi105 ceph-mon[34795]: pgmap v499: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:38:20.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:20 smithi110 ceph-mon[35497]: pgmap v499: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:38:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:20 smithi050 ceph-mon[31739]: pgmap v499: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 488 op/s 2024-05-17T04:38:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:22 smithi110 ceph-mon[35497]: pgmap v500: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:38:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:22 smithi050 ceph-mon[31739]: pgmap v500: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:38:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:22 smithi105 ceph-mon[34795]: pgmap v500: 129 pgs: 129 active+clean; 777 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:38:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:24 smithi110 ceph-mon[35497]: pgmap v501: 129 pgs: 129 active+clean; 777 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 484 op/s 2024-05-17T04:38:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:24 smithi050 ceph-mon[31739]: pgmap v501: 129 pgs: 129 active+clean; 777 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 484 op/s 2024-05-17T04:38:24.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:24 smithi105 ceph-mon[34795]: pgmap v501: 129 pgs: 129 active+clean; 777 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 484 op/s 2024-05-17T04:38:26.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:26 smithi050 ceph-mon[31739]: pgmap v502: 129 pgs: 129 active+clean; 785 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-17T04:38:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:26 smithi105 ceph-mon[34795]: pgmap v502: 129 pgs: 129 active+clean; 785 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-17T04:38:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:26 smithi110 ceph-mon[35497]: pgmap v502: 129 pgs: 129 active+clean; 785 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-17T04:38:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:28 smithi050 ceph-mon[31739]: pgmap v503: 129 pgs: 129 active+clean; 785 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-05-17T04:38:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:28 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:38:28.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:28 smithi105 ceph-mon[34795]: pgmap v503: 129 pgs: 129 active+clean; 785 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-05-17T04:38:28.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:28 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:38:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:28 smithi110 ceph-mon[35497]: pgmap v503: 129 pgs: 129 active+clean; 785 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 419 op/s 2024-05-17T04:38:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:28 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:38:29.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:38:29.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:38:29.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:38:29.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:38:29.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:38:29.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:38:29.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:38:29.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:38:29.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:38:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:30 smithi050 ceph-mon[31739]: pgmap v504: 129 pgs: 129 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-17T04:38:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:30 smithi105 ceph-mon[34795]: pgmap v504: 129 pgs: 129 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-17T04:38:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:30 smithi110 ceph-mon[35497]: pgmap v504: 129 pgs: 129 active+clean; 793 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 374 op/s 2024-05-17T04:38:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:32 smithi050 ceph-mon[31739]: pgmap v505: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 -- 172.21.15.50:0/736920124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 msgr2=0x7f198c10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 --2- 172.21.15.50:0/736920124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c10b350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1984009be0 tx=0x7f198402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 -- 172.21.15.50:0/736920124 shutdown_connections 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 --2- 172.21.15.50:0/736920124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c10b350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 --2- 172.21.15.50:0/736920124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 0x7f198c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 --2- 172.21.15.50:0/736920124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 -- 172.21.15.50:0/736920124 >> 172.21.15.50:0/736920124 conn(0x7f198c0fd880 msgr2=0x7f198c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 -- 172.21.15.50:0/736920124 shutdown_connections 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.446+0000 7f1993d44640 1 -- 172.21.15.50:0/736920124 wait complete. 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 Processor -- start 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 -- start start 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 0x7f198c101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10eb50 con 0x7f198c1065d0 2024-05-17T04:38:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10e9d0 con 0x7f198c108f30 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1993d44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10ecd0 con 0x7f198c103c70 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:43114/0 (socket says 172.21.15.50:43114) 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f1991ae0640 1 -- 172.21.15.50:0/2086105379 learned_addr learned my addr 172.21.15.50:0/2086105379 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.447+0000 7f19912df640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 0x7f198c101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f19922e1640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 -- 172.21.15.50:0/2086105379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 msgr2=0x7f198c1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 -- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 msgr2=0x7f198c101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 0x7f198c101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 -- 172.21.15.50:0/2086105379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c1027f0 con 0x7f198c103c70 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1991ae0640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f198800e9c0 tx=0x7f198800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1982ffd640 1 -- 172.21.15.50:0/2086105379 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800cde0 con 0x7f198c103c70 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1982ffd640 1 -- 172.21.15.50:0/2086105379 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19880108c0 con 0x7f198c103c70 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c1a3820 con 0x7f198c103c70 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1982ffd640 1 -- 172.21.15.50:0/2086105379 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988018870 con 0x7f198c103c70 2024-05-17T04:38:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.448+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1a3ce0 con 0x7f198c103c70 2024-05-17T04:38:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.452+0000 7f1982ffd640 1 -- 172.21.15.50:0/2086105379 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f198801f020 con 0x7f198c103c70 2024-05-17T04:38:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.452+0000 7f199abfd740 1 Processor -- start 2024-05-17T04:38:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.452+0000 7f199abfd740 1 -- start start 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 0x562f9ee7fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee86c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f9edf1cc0 con 0x562f9eabb4d0 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f9ede1860 con 0x562f9eab32f0 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f199abfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f9ede58a0 con 0x562f9eadc1b0 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:43120/0 (socket says 172.21.15.50:43120) 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19912df640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee86c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 -- 172.21.15.50:0/2421525219 learned_addr learned my addr 172.21.15.50:0/2421525219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f1991ae0640 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 0x562f9ee7fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f1982ffd640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 0x7f19780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 -- 172.21.15.50:0/2421525219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 msgr2=0x562f9ee7fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 0x562f9ee7fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 -- 172.21.15.50:0/2421525219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 msgr2=0x562f9ee86c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee86c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.453+0000 7f19922e1640 1 -- 172.21.15.50:0/2421525219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9eda9d50 con 0x562f9eadc1b0 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f1982ffd640 1 -- 172.21.15.50:0/2086105379 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f19880e27d0 con 0x7f198c103c70 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f1991ae0640 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 0x562f9ee7fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f19922e1640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 0x7f19780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f19922e1640 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1984009be0 tx=0x7f1984030480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c3262fe6f45452 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f1977fff640 1 -- 172.21.15.50:0/2421525219 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984036040 con 0x562f9eadc1b0 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f1977fff640 1 -- 172.21.15.50:0/2421525219 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1984037040 con 0x562f9eadc1b0 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.454+0000 7f1977fff640 1 -- 172.21.15.50:0/2421525219 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984049640 con 0x562f9eadc1b0 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f19922e1640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 0x7f19780b4cf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f198404b000 tx=0x7f1984048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- 172.21.15.50:0/2421525219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 msgr2=0x562f9ee7f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1984009be0 tx=0x7f1984030480 comp rx=0 tx=0).stop 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- 172.21.15.50:0/2421525219 shutdown_connections 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee86c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eab32f0 0x562f9ee7fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 --2- 172.21.15.50:0/2421525219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eadc1b0 0x562f9ee7f770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- 172.21.15.50:0/2421525219 >> 172.21.15.50:0/2421525219 conn(0x562f9ead0300 msgr2=0x562f9ee7b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- 172.21.15.50:0/2421525219 shutdown_connections 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- 172.21.15.50:0/2421525219 wait complete. 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.456+0000 7f199abfd740 1 -- start start 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.457+0000 7f199abfd740 1 Processor -- start 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.457+0000 7f199abfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.457+0000 7f199abfd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f19912df640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:43132/0 (socket says 172.21.15.50:43132) 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f199abfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 0x562f9ee90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f199abfd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f9ed1dab0 con 0x562f9eabb4d0 2024-05-17T04:38:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f199abfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f9ed9e4d0 con 0x562f9eadc1b0 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f199abfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f9ee87690 con 0x562f9eab32f0 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 -- 172.21.15.50:0/4234437398 learned_addr learned my addr 172.21.15.50:0/4234437398 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f199abfd740 10 client.0 _ll_register_callbacks cb 0x562f9eddc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f19922e1640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 0x562f9ee90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 -- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 msgr2=0x562f9ee90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 0x562f9ee90500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 -- 172.21.15.50:0/4234437398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 msgr2=0x562f9ee895a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee895a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 -- 172.21.15.50:0/4234437398 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9ed9f340 con 0x562f9eab32f0 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f19922e1640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 0x562f9ee90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:38:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f1991ae0640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19880a5ae0 tx=0x7f19880b22c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d687128fa18e3885 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f19757fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.458+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880a9a60 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f19880b2430 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1984037040 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19880ade10 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 10 client.25225 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f9eb47790 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1984037040 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f19880a8cc0 con 0x562f9eab32f0 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 10 client.25225 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 10 client.25225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 20 client.25225 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea9ad80 0x562f9ee96770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 4 client.25225 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.459+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f9edf3c10 con 0x562f9ea9ad80 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.460+0000 7f19912df640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea9ad80 0x562f9ee96770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.460+0000 7f19757fa640 10 client.25225 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.460+0000 7f19912df640 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea9ad80 0x562f9ee96770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492f70116e10ab3a server_cookie=26aa514754d37e7b in_seq=0 out_seq=0 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f19757fa640 1 -- 172.21.15.50:0/4234437398 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f9edf3c10 con 0x562f9ea9ad80 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f19757fa640 10 client.25225 handle_command_reply: tid=0 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 client.25225 shutdown 2024-05-17T04:38:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 msgr2=0x562f9ee89060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19880a5ae0 tx=0x7f19880b22c0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 shutdown_connections 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eadc1b0 0x562f9ee90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea9ad80 0x562f9ee96770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eabb4d0 0x562f9ee895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 --2- 172.21.15.50:0/4234437398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee89060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 >> 172.21.15.50:0/4234437398 conn(0x562f9ead0300 msgr2=0x562f9ee7a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f19757fa640 0 client.25225 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 shutdown_connections 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.461+0000 7f199abfd740 1 -- 172.21.15.50:0/4234437398 wait complete. 2024-05-17T04:38:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.462+0000 7f199abfd740 20 client.25225 trim_cache size 0 max 16384 2024-05-17T04:38:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 Processor -- start 2024-05-17T04:38:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 -- start start 2024-05-17T04:38:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee7b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9ee7dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1b0a30 con 0x562f9eadc1b0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1ae2a0 con 0x562f9eabb4d0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f199abfd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1ac0f0 con 0x562f9eab32f0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.484+0000 7f19922e1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9ee7dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee7b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19912df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee7b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:43140/0 (socket says 172.21.15.50:43140) 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19912df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46630/0 (socket says 172.21.15.50:46630) 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f1991ae0640 1 -- 172.21.15.50:0/2879352506 learned_addr learned my addr 172.21.15.50:0/2879352506 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 -- 172.21.15.50:0/2879352506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 msgr2=0x562f9ee7b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee7b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 -- 172.21.15.50:0/2879352506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 msgr2=0x562f9ee90420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 -- 172.21.15.50:0/2879352506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f9ebc0620 con 0x562f9eadc1b0 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19912df640 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19922e1640 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9ee7dea0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1984030ab0 tx=0x7f1984051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384bee9a210cc535 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.485+0000 7f19777fe640 1 -- 172.21.15.50:0/2879352506 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984036030 con 0x562f9eadc1b0 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.486+0000 7f19777fe640 1 -- 172.21.15.50:0/2879352506 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1984049d70 con 0x562f9eadc1b0 2024-05-17T04:38:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.486+0000 7f19777fe640 1 -- 172.21.15.50:0/2879352506 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984004040 con 0x562f9eadc1b0 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- 172.21.15.50:0/2879352506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 msgr2=0x562f9ee7dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9ee7dea0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1984030ab0 tx=0x7f1984051000 comp rx=0 tx=0).stop 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- 172.21.15.50:0/2879352506 shutdown_connections 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9ee7dea0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee90420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 --2- 172.21.15.50:0/2879352506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee7b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- 172.21.15.50:0/2879352506 >> 172.21.15.50:0/2879352506 conn(0x562f9ead0300 msgr2=0x562f9ee7aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- 172.21.15.50:0/2879352506 shutdown_connections 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- 172.21.15.50:0/2879352506 wait complete. 2024-05-17T04:38:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.487+0000 7f199abfd740 1 -- start start 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f199abfd740 1 Processor -- start 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f199abfd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f199abfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f199abfd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9eefcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f199abfd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1ac760 con 0x562f9eadc1b0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f19912df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f19912df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46634/0 (socket says 172.21.15.50:46634) 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f1991ae0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.488+0000 7f19912df640 1 -- 172.21.15.50:0/3066726348 learned_addr learned my addr 172.21.15.50:0/3066726348 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19922e1640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9eefcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1adbd0 con 0x562f9eabb4d0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1ace30 con 0x562f9eab32f0 2024-05-17T04:38:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f199abfd740 10 client.0 _ll_register_callbacks cb 0x7f19880a9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 -- 172.21.15.50:0/3066726348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 msgr2=0x562f9ee8d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee8d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 -- 172.21.15.50:0/3066726348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 msgr2=0x562f9eefcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9eefcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f9ee906d0 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19912df640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f197c009530 tx=0x7f197c0204d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3753facbf87aac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19757fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c004220 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f197c0043c0 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.489+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1984049d70 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f197c031050 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 10 client.25241 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f9eefe100 con 0x562f9eabb4d0 2024-05-17T04:38:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1984049d70 con 0x562f9eabb4d0 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f197c0038f0 con 0x562f9eabb4d0 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 10 client.25241 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 20 client.25241 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea701c0 0x562f9ef00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 4 client.25241 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.490+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f9eded3f0 con 0x562f9ea701c0 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.491+0000 7f1991ae0640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea701c0 0x562f9ef00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.491+0000 7f19757fa640 10 client.25241 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.491+0000 7f1991ae0640 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea701c0 0x562f9ef00b80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac3c83a8c108cc3 server_cookie=e5f93e5904f74ec2 in_seq=0 out_seq=0 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f19757fa640 1 -- 172.21.15.50:0/3066726348 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562f9eded3f0 con 0x562f9ea701c0 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f19757fa640 10 client.25241 handle_command_reply: tid=0 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 client.25241 shutdown 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 msgr2=0x562f9ee8d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f197c009530 tx=0x7f197c0204d0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 shutdown_connections 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562f9ea701c0 0x562f9ef00b80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f9eadc1b0 0x562f9eefcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.501+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f9eabb4d0 0x562f9ee8d750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f199abfd740 1 --2- 172.21.15.50:0/3066726348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f9eab32f0 0x562f9ee8d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f19757fa640 0 client.25241 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 >> 172.21.15.50:0/3066726348 conn(0x562f9ead0300 msgr2=0x562f9ee797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 shutdown_connections 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f199abfd740 1 -- 172.21.15.50:0/3066726348 wait complete. 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f199abfd740 20 client.25241 trim_cache size 0 max 16384 2024-05-17T04:38:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:38:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:38:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:38:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 msgr2=0x7f19780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 0x7f19780b4cf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f198404b000 tx=0x7f1984048000 comp rx=0 tx=0).stop 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 msgr2=0x7f198c1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.502+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f198800e9c0 tx=0x7f198800ee80 comp rx=0 tx=0).stop 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 shutdown_connections 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f198c108f30 0x7f198c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f19780b27e0 0x7f19780b4cf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f198c1065d0 0x7f198c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 --2- 172.21.15.50:0/2086105379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f198c103c70 0x7f198c1034e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 >> 172.21.15.50:0/2086105379 conn(0x7f198c0fd880 msgr2=0x7f198c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 shutdown_connections 2024-05-17T04:38:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:32.503+0000 7f1993d44640 1 -- 172.21.15.50:0/2086105379 wait complete. 2024-05-17T04:38:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:32 smithi105 ceph-mon[34795]: pgmap v505: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-17T04:38:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:32 smithi110 ceph-mon[35497]: pgmap v505: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 409 op/s 2024-05-17T04:38:34.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:34 smithi105 ceph-mon[34795]: pgmap v506: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T04:38:34.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:34 smithi110 ceph-mon[35497]: pgmap v506: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T04:38:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:34 smithi050 ceph-mon[31739]: pgmap v506: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 530 op/s 2024-05-17T04:38:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:36 smithi105 ceph-mon[34795]: pgmap v507: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-17T04:38:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:36 smithi110 ceph-mon[35497]: pgmap v507: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-17T04:38:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:36 smithi050 ceph-mon[31739]: pgmap v507: 129 pgs: 129 active+clean; 793 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-05-17T04:38:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:38 smithi105 ceph-mon[34795]: pgmap v508: 129 pgs: 129 active+clean; 793 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-05-17T04:38:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:38 smithi110 ceph-mon[35497]: pgmap v508: 129 pgs: 129 active+clean; 793 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-05-17T04:38:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:38 smithi050 ceph-mon[31739]: pgmap v508: 129 pgs: 129 active+clean; 793 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 548 op/s 2024-05-17T04:38:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:40 smithi105 ceph-mon[34795]: pgmap v509: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 504 op/s 2024-05-17T04:38:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:40 smithi110 ceph-mon[35497]: pgmap v509: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 504 op/s 2024-05-17T04:38:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:40 smithi050 ceph-mon[31739]: pgmap v509: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 504 op/s 2024-05-17T04:38:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:42 smithi105 ceph-mon[34795]: pgmap v510: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 476 op/s 2024-05-17T04:38:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:42 smithi110 ceph-mon[35497]: pgmap v510: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 476 op/s 2024-05-17T04:38:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:42 smithi050 ceph-mon[31739]: pgmap v510: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 476 op/s 2024-05-17T04:38:44.190 INFO:teuthology.orchestra.run.smithi050.stderr:creating primary keys... 2024-05-17T04:38:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:44 smithi105 ceph-mon[34795]: pgmap v511: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:38:44.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:44 smithi110 ceph-mon[35497]: pgmap v511: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:38:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:44 smithi050 ceph-mon[31739]: pgmap v511: 129 pgs: 129 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 -- 172.21.15.50:0/2725937206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4108f30 msgr2=0x7fc1a410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/2725937206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4108f30 0x7fc1a410b350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009900 tx=0x7fc19c02a410 comp rx=0 tx=0).stop 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 -- 172.21.15.50:0/2725937206 shutdown_connections 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/2725937206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4108f30 0x7fc1a410b350 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/2725937206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 0x7fc1a41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/2725937206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a4103c70 0x7fc1a4106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 -- 172.21.15.50:0/2725937206 >> 172.21.15.50:0/2725937206 conn(0x7fc1a40fd880 msgr2=0x7fc1a40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 -- 172.21.15.50:0/2725937206 shutdown_connections 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.619+0000 7fc1ac19a640 1 -- 172.21.15.50:0/2725937206 wait complete. 2024-05-17T04:38:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 Processor -- start 2024-05-17T04:38:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 -- start start 2024-05-17T04:38:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 0x7fc1a4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a4108f30 0x7fc1a41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a410eba0 con 0x7fc1a4103c70 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a410ea20 con 0x7fc1a41065d0 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1ac19a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a410ed20 con 0x7fc1a4108f30 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1a9f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1a9f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34424/0 (socket says 172.21.15.50:34424) 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.620+0000 7fc1a9f36640 1 -- 172.21.15.50:0/621539177 learned_addr learned my addr 172.21.15.50:0/621539177 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9735640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 0x7fc1a4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 -- 172.21.15.50:0/621539177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a4108f30 msgr2=0x7fc1a41a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a4108f30 0x7fc1a41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 -- 172.21.15.50:0/621539177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 msgr2=0x7fc1a4199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 0x7fc1a4199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 -- 172.21.15.50:0/621539177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a41a12d0 con 0x7fc1a4103c70 2024-05-17T04:38:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000eb70 tx=0x7fc1a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc19affd640 1 -- 172.21.15.50:0/621539177 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0003b70 con 0x7fc1a4103c70 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a41a15c0 con 0x7fc1a4103c70 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc19affd640 1 -- 172.21.15.50:0/621539177 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1a000ee70 con 0x7fc1a4103c70 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc19affd640 1 -- 172.21.15.50:0/621539177 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0010690 con 0x7fc1a4103c70 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.621+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a41a1ac0 con 0x7fc1a4103c70 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.624+0000 7fc1b3055740 1 Processor -- start 2024-05-17T04:38:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 -- start start 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x560296142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602960b4cc0 con 0x560295d7e4d0 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5602960a4860 con 0x560295d762f0 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1b3055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5602960a88a0 con 0x560295d9f1b0 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc19affd640 1 -- 172.21.15.50:0/621539177 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1a0004020 con 0x7fc1a4103c70 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1a9735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1a9735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34434/0 (socket says 172.21.15.50:34434) 2024-05-17T04:38:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1a9735640 1 -- 172.21.15.50:0/2106112553 learned_addr learned my addr 172.21.15.50:0/2106112553 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1aa737640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x560296142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.625+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 -- 172.21.15.50:0/2106112553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 msgr2=0x560296142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x560296142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 -- 172.21.15.50:0/2106112553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 msgr2=0x560296142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 -- 172.21.15.50:0/2106112553 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029606cd50 con 0x560295d7e4d0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1a9735640 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bc60 tx=0x7fc19400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea014e684fbd572 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc18ffff640 1 -- 172.21.15.50:0/2106112553 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194015070 con 0x560295d7e4d0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc18ffff640 1 -- 172.21.15.50:0/2106112553 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc19401a8f0 con 0x560295d7e4d0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc18ffff640 1 -- 172.21.15.50:0/2106112553 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194011800 con 0x560295d7e4d0 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc19affd640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 0x7fc1900b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc1aa737640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 0x7fc1900b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.626+0000 7fc19affd640 1 -- 172.21.15.50:0/621539177 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc1a00e2d80 con 0x7fc1a4103c70 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 -- 172.21.15.50:0/2106112553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 msgr2=0x560296149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1aa737640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 0x7fc1900b4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560296143960 tx=0x7fc19c005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bc60 tx=0x7fc19400f730 comp rx=0 tx=0).stop 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 -- 172.21.15.50:0/2106112553 shutdown_connections 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x560296149c10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 --2- 172.21.15.50:0/2106112553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x560296142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 -- 172.21.15.50:0/2106112553 >> 172.21.15.50:0/2106112553 conn(0x560295d93300 msgr2=0x56029613e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 -- 172.21.15.50:0/2106112553 shutdown_connections 2024-05-17T04:38:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.627+0000 7fc1b3055740 1 -- 172.21.15.50:0/2106112553 wait complete. 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 -- start start 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 Processor -- start 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d762f0 0x56029614c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560295fe0ab0 con 0x560295d9f1b0 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5602960614d0 con 0x560295d7e4d0 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56029614a690 con 0x560295d762f0 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1b3055740 10 client.0 _ll_register_callbacks cb 0x56029609f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1aa737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1aa737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34442/0 (socket says 172.21.15.50:34442) 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1aa737640 1 -- 172.21.15.50:0/1406466639 learned_addr learned my addr 172.21.15.50:0/1406466639 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.628+0000 7fc1a9735640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d762f0 0x56029614c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 -- 172.21.15.50:0/1406466639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d762f0 msgr2=0x56029614c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d762f0 0x56029614c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 -- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 msgr2=0x56029614c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 -- 172.21.15.50:0/1406466639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560296062340 con 0x560295d9f1b0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1aa737640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009900 tx=0x7fc19c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1aeb145a14c824 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c04d070 con 0x560295d9f1b0 2024-05-17T04:38:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc19c003d00 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19401a8f0 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc19c059050 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1b3055740 10 client.15477 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.629+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560295e0a790 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc19401a8f0 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc19c0494c0 con 0x560295d9f1b0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 10 client.15477 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 20 client.15477 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d5dd80 0x560296159870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 4 client.15477 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602960b6c10 con 0x560295d5dd80 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.630+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d5dd80 0x560296159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.631+0000 7fc18d7fa640 10 client.15477 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.631+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d5dd80 0x560296159870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04c39d977a6cdfe server_cookie=997fcbc1c4379513 in_seq=0 out_seq=0 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.631+0000 7fc18d7fa640 1 -- 172.21.15.50:0/1406466639 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5602960b6c10 con 0x560295d5dd80 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.631+0000 7fc18d7fa640 10 client.15477 handle_command_reply: tid=0 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 client.15477 shutdown 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 msgr2=0x5602961534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009900 tx=0x7fc19c009260 comp rx=0 tx=0).stop 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 shutdown_connections 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d5dd80 0x560296159870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d9f1b0 0x5602961534e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1406466639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d762f0 0x56029614c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc18d7fa640 0 client.15477 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 >> 172.21.15.50:0/1406466639 conn(0x560295d93300 msgr2=0x56029613d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 shutdown_connections 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 1 -- 172.21.15.50:0/1406466639 wait complete. 2024-05-17T04:38:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.632+0000 7fc1b3055740 20 client.15477 trim_cache size 0 max 16384 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.647+0000 7fc1b3055740 1 Processor -- start 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.647+0000 7fc1b3055740 1 -- start start 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296138f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029613e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ae8d0 con 0x560295d7e4d0 2024-05-17T04:38:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ac140 con 0x560295d762f0 2024-05-17T04:38:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1b3055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41a9f90 con 0x560295d9f1b0 2024-05-17T04:38:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34450/0 (socket says 172.21.15.50:34450) 2024-05-17T04:38:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 -- 172.21.15.50:0/1302453004 learned_addr learned my addr 172.21.15.50:0/1302453004 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296138f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1aa737640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029613e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 -- 172.21.15.50:0/1302453004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 msgr2=0x56029613e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029613e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 -- 172.21.15.50:0/1302453004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 msgr2=0x560296138f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296138f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 -- 172.21.15.50:0/1302453004 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560295e83620 con 0x560295d7e4d0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296138f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1aa737640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029613e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc1a9735640 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bf90 tx=0x7fc19400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9566c72e47a4c3bc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.648+0000 7fc18f7fe640 1 -- 172.21.15.50:0/1302453004 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19402a070 con 0x560295d7e4d0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc18f7fe640 1 -- 172.21.15.50:0/1302453004 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc194004680 con 0x560295d7e4d0 2024-05-17T04:38:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc18f7fe640 1 -- 172.21.15.50:0/1302453004 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194015070 con 0x560295d7e4d0 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 -- 172.21.15.50:0/1302453004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 msgr2=0x56029613daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc19400bf90 tx=0x7fc19400b7a0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 -- 172.21.15.50:0/1302453004 shutdown_connections 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029613e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d7e4d0 0x56029613daf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 --2- 172.21.15.50:0/1302453004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d762f0 0x560296138f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 -- 172.21.15.50:0/1302453004 >> 172.21.15.50:0/1302453004 conn(0x560295d93300 msgr2=0x56029614aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 -- 172.21.15.50:0/1302453004 shutdown_connections 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.649+0000 7fc1b3055740 1 -- 172.21.15.50:0/1302453004 wait complete. 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 -- start start 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 Processor -- start 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029614fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41aa600 con 0x560295d762f0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41aba70 con 0x560295d7e4d0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41aacd0 con 0x560295d9f1b0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1b3055740 10 client.0 _ll_register_callbacks cb 0x7fc19c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1a9f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1a9f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34460/0 (socket says 172.21.15.50:34460) 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.650+0000 7fc1a9f36640 1 -- 172.21.15.50:0/3144974639 learned_addr learned my addr 172.21.15.50:0/3144974639 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9735640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1aa737640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029614fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 -- 172.21.15.50:0/3144974639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 msgr2=0x56029614fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029614fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 -- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 msgr2=0x56029614f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 -- 172.21.15.50:0/3144974639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602961bfc10 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1aa737640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029614fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9735640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1a9f36640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0003de0 tx=0x7fc1a00a5fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aca72a7b747357 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00c3070 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc1a0004310 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194004680 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc1a00ae3c0 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1b3055740 10 client.15483 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5602961c0cb0 con 0x560295d762f0 2024-05-17T04:38:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.651+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc194004680 con 0x560295d762f0 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc1a00a5980 con 0x560295d762f0 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 10 client.15483 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 20 client.15483 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d331c0 0x5602961c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 4 client.15483 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:38:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602960b03f0 con 0x560295d331c0 2024-05-17T04:38:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.652+0000 7fc1a9735640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d331c0 0x5602961c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.653+0000 7fc1a9735640 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d331c0 0x5602961c3700 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8795e973e7ce2e27 server_cookie=1624ac082aa26c53 in_seq=0 out_seq=0 2024-05-17T04:38:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.653+0000 7fc18d7fa640 10 client.15483 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc18d7fa640 1 -- 172.21.15.50:0/3144974639 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5602960b03f0 con 0x560295d331c0 2024-05-17T04:38:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc18d7fa640 10 client.15483 handle_command_reply: tid=0 2024-05-17T04:38:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 client.15483 shutdown 2024-05-17T04:38:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 msgr2=0x560296152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0003de0 tx=0x7fc1a00a5fb0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 shutdown_connections 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560295d9f1b0 0x56029614fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560295d7e4d0 0x56029614f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560295d331c0 0x5602961c3700 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 --2- 172.21.15.50:0/3144974639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560295d762f0 0x560296152940 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc18d7fa640 0 client.15483 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 >> 172.21.15.50:0/3144974639 conn(0x560295d93300 msgr2=0x560296138f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 shutdown_connections 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 1 -- 172.21.15.50:0/3144974639 wait complete. 2024-05-17T04:38:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.668+0000 7fc1b3055740 20 client.15483 trim_cache size 0 max 16384 2024-05-17T04:38:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:38:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:38:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:38:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 msgr2=0x7fc1900b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 0x7fc1900b4cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560296143960 tx=0x7fc19c005720 comp rx=0 tx=0).stop 2024-05-17T04:38:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 msgr2=0x7fc1a4199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000eb70 tx=0x7fc1a000c6a0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 shutdown_connections 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1a4108f30 0x7fc1a41a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1a41065d0 0x7fc1a4199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc1900b27e0 0x7fc1900b4cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 --2- 172.21.15.50:0/621539177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc1a4103c70 0x7fc1a4199790 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 >> 172.21.15.50:0/621539177 conn(0x7fc1a40fd880 msgr2=0x7fc1a4104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 shutdown_connections 2024-05-17T04:38:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:45.669+0000 7fc1ac19a640 1 -- 172.21.15.50:0/621539177 wait complete. 2024-05-17T04:38:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:46 smithi105 ceph-mon[34795]: pgmap v512: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 502 op/s 2024-05-17T04:38:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:46 smithi110 ceph-mon[35497]: pgmap v512: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 502 op/s 2024-05-17T04:38:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:46 smithi050 ceph-mon[31739]: pgmap v512: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 502 op/s 2024-05-17T04:38:48.544 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:48 smithi110 ceph-mon[35497]: pgmap v513: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:38:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:48 smithi105 ceph-mon[34795]: pgmap v513: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:38:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:48 smithi050 ceph-mon[31739]: pgmap v513: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 454 op/s 2024-05-17T04:38:50.378 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:50 smithi050 ceph-mon[31739]: pgmap v514: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2024-05-17T04:38:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:50 smithi105 ceph-mon[34795]: pgmap v514: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2024-05-17T04:38:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:50 smithi110 ceph-mon[35497]: pgmap v514: 129 pgs: 129 active+clean; 793 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2024-05-17T04:38:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:52 smithi105 ceph-mon[34795]: pgmap v515: 129 pgs: 129 active+clean; 825 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2024-05-17T04:38:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:52 smithi110 ceph-mon[35497]: pgmap v515: 129 pgs: 129 active+clean; 825 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2024-05-17T04:38:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:52 smithi050 ceph-mon[31739]: pgmap v515: 129 pgs: 129 active+clean; 825 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2024-05-17T04:38:54.508 INFO:teuthology.orchestra.run.smithi050.stderr:done in 162.03 s (drop tables 0.01 s, create tables 0.09 s, client-side generate 35.11 s, vacuum 116.50 s, primary keys 10.32 s). 2024-05-17T04:38:54.509 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:54 smithi050 ceph-mon[31739]: pgmap v516: 129 pgs: 129 active+clean; 867 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 315 op/s 2024-05-17T04:38:54.512 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-17T04:38:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:54 smithi105 ceph-mon[34795]: pgmap v516: 129 pgs: 129 active+clean; 867 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 315 op/s 2024-05-17T04:38:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:54 smithi110 ceph-mon[35497]: pgmap v516: 129 pgs: 129 active+clean; 867 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 315 op/s 2024-05-17T04:38:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:56 smithi105 ceph-mon[34795]: pgmap v517: 129 pgs: 129 active+clean; 928 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 20 MiB/s wr, 101 op/s 2024-05-17T04:38:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:56 smithi110 ceph-mon[35497]: pgmap v517: 129 pgs: 129 active+clean; 928 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 20 MiB/s wr, 101 op/s 2024-05-17T04:38:56.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:56 smithi050 ceph-mon[31739]: pgmap v517: 129 pgs: 129 active+clean; 928 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 20 MiB/s wr, 101 op/s 2024-05-17T04:38:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:starting vacuum...end. 2024-05-17T04:38:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:38:58 smithi105 ceph-mon[34795]: pgmap v518: 129 pgs: 129 active+clean; 946 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-17T04:38:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:38:58 smithi110 ceph-mon[35497]: pgmap v518: 129 pgs: 129 active+clean; 946 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-17T04:38:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:38:58 smithi050 ceph-mon[31739]: pgmap v518: 129 pgs: 129 active+clean; 946 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 33 op/s 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/1359658373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f08103c80 msgr2=0x7f0f081060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/1359658373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f08103c80 0x7f0f081060a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009900 tx=0x7f0ef802a410 comp rx=0 tx=0).stop 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/1359658373 shutdown_connections 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/1359658373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 0x7f0f0810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/1359658373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f081065e0 0x7f0f08108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/1359658373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f08103c80 0x7f0f081060a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/1359658373 >> 172.21.15.50:0/1359658373 conn(0x7f0f080fd8b0 msgr2=0x7f0f080ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/1359658373 shutdown_connections 2024-05-17T04:38:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.786+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/1359658373 wait complete. 2024-05-17T04:38:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 Processor -- start 2024-05-17T04:38:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 -- start start 2024-05-17T04:38:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 0x7f0f08101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 0x7f0f081020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810eb60 con 0x7f0f081065e0 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810e9e0 con 0x7f0f08103c80 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f0e1bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810ece0 con 0x7f0f08108f40 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 0x7f0f08101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:47272/0 (socket says 172.21.15.50:47272) 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f077fe640 1 -- 172.21.15.50:0/2373773769 learned_addr learned my addr 172.21.15.50:0/2373773769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 0x7f0f08101b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57394/0 (socket says 172.21.15.50:57394) 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.787+0000 7f0f07fff640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 0x7f0f081020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 -- 172.21.15.50:0/2373773769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 msgr2=0x7f0f081020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 0x7f0f081020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 -- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 msgr2=0x7f0f08101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 0x7f0f08101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 -- 172.21.15.50:0/2373773769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08102800 con 0x7f0f08103c80 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f077fe640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8031460 tx=0x7f0ef8005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f057fa640 1 -- 172.21.15.50:0/2373773769 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef803f030 con 0x7f0f08103c80 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f081a38e0 con 0x7f0f08103c80 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f057fa640 1 -- 172.21.15.50:0/2373773769 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef80043d0 con 0x7f0f08103c80 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f057fa640 1 -- 172.21.15.50:0/2373773769 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8035070 con 0x7f0f08103c80 2024-05-17T04:38:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.788+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a3ce0 con 0x7f0f08103c80 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 Processor -- start 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 -- start start 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557981747cc0 con 0x5579814321b0 2024-05-17T04:38:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557981737860 con 0x5579814092f0 2024-05-17T04:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f15073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55798173b8a0 con 0x5579814114d0 2024-05-17T04:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57404/0 (socket says 172.21.15.50:57404) 2024-05-17T04:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0f07fff640 1 -- 172.21.15.50:0/1355615857 learned_addr learned my addr 172.21.15.50:0/1355615857 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.791+0000 7f0efffff640 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 -- 172.21.15.50:0/1355615857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 msgr2=0x5579817d5cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d5cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 -- 172.21.15.50:0/1355615857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 msgr2=0x5579817d5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 -- 172.21.15.50:0/1355615857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579816ffd50 con 0x5579814114d0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0f057fa640 1 -- 172.21.15.50:0/2373773769 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0ef8042030 con 0x7f0f08103c80 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efffff640 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817dcc10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009bf0 tx=0x7f0ef0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857cc762f01ba016 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efdffb640 1 -- 172.21.15.50:0/1355615857 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001e070 con 0x5579814114d0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efdffb640 1 -- 172.21.15.50:0/1355615857 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef00035f0 con 0x5579814114d0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.792+0000 7f0efdffb640 1 -- 172.21.15.50:0/1355615857 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0019690 con 0x5579814114d0 2024-05-17T04:38:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f057fa640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 0x7f0ef40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f07fff640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 0x7f0ef40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f057fa640 1 -- 172.21.15.50:0/2373773769 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0ef8102980 con 0x7f0f08103c80 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 -- 172.21.15.50:0/1355615857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 msgr2=0x5579817dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817dcc10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009bf0 tx=0x7f0ef0011810 comp rx=0 tx=0).stop 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 -- 172.21.15.50:0/1355615857 shutdown_connections 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817dcc10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 --2- 172.21.15.50:0/1355615857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814321b0 0x5579817d5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 -- 172.21.15.50:0/1355615857 >> 172.21.15.50:0/1355615857 conn(0x557981426300 msgr2=0x5579817d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.793+0000 7f0f15073740 1 -- 172.21.15.50:0/1355615857 shutdown_connections 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.794+0000 7f0f07fff640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 0x7f0ef40b4cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5579817d6960 tx=0x7f0f00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.794+0000 7f0f15073740 1 -- 172.21.15.50:0/1355615857 wait complete. 2024-05-17T04:38:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.794+0000 7f0f15073740 1 -- start start 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.794+0000 7f0f15073740 1 Processor -- start 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817df610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x5579817e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557981673ab0 con 0x5579814092f0 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5579816f44d0 con 0x5579814321b0 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5579817dd690 con 0x5579814114d0 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57418/0 (socket says 172.21.15.50:57418) 2024-05-17T04:38:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f077fe640 1 -- 172.21.15.50:0/1814898282 learned_addr learned my addr 172.21.15.50:0/1814898282 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x5579817e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f15073740 10 client.0 _ll_register_callbacks cb 0x557981732950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0efffff640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817df610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 -- 172.21.15.50:0/1814898282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 msgr2=0x5579817df610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817df610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 -- 172.21.15.50:0/1814898282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 msgr2=0x5579817df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f07fff640 1 -- 172.21.15.50:0/1814898282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579816f5340 con 0x5579814321b0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.795+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x5579817e6570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000f790 tx=0x7f0f00004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f49bdef07b955d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004b20 con 0x5579814321b0 2024-05-17T04:38:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef00035f0 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0f00004cc0 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f00004310 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0f15073740 10 client.25259 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.796+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55798149d790 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef00035f0 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0f00026070 con 0x5579814321b0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 10 client.25259 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 20 client.25259 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813f0d80 0x5579817ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 4 client.25259 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557981749c10 con 0x5579813f0d80 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.797+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813f0d80 0x5579817ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.798+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813f0d80 0x5579817ec9e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea22510f3da30d4 server_cookie=13d626fbd1da8ed8 in_seq=0 out_seq=0 2024-05-17T04:38:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.798+0000 7f0ee6ffd640 10 client.25259 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1814898282 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557981749c10 con 0x5579813f0d80 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0ee6ffd640 10 client.25259 handle_command_reply: tid=0 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 client.25259 shutdown 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 msgr2=0x5579817e6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x5579817e6570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000f790 tx=0x7f0f00004ab0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 shutdown_connections 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x5579817e6570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814114d0 0x5579817df610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813f0d80 0x5579817ec9e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 --2- 172.21.15.50:0/1814898282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814092f0 0x5579817df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 >> 172.21.15.50:0/1814898282 conn(0x557981426300 msgr2=0x5579817d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0ee6ffd640 0 client.25259 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 shutdown_connections 2024-05-17T04:38:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 1 -- 172.21.15.50:0/1814898282 wait complete. 2024-05-17T04:38:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.799+0000 7f0f15073740 20 client.25259 trim_cache size 0 max 16384 2024-05-17T04:38:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.818+0000 7f0f15073740 1 Processor -- start 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.818+0000 7f0f15073740 1 -- start start 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814321b0 0x5579817d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081b0a60 con 0x5579814114d0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ae2d0 con 0x5579814092f0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f15073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ac120 con 0x5579814321b0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57434/0 (socket says 172.21.15.50:57434) 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0efffff640 1 -- 172.21.15.50:0/1751719379 learned_addr learned my addr 172.21.15.50:0/1751719379 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814321b0 0x5579817d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 -- 172.21.15.50:0/1751719379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814321b0 msgr2=0x5579817d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814321b0 0x5579817d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 -- 172.21.15.50:0/1751719379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 msgr2=0x5579817d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0f077fe640 1 -- 172.21.15.50:0/1751719379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557981516620 con 0x5579814092f0 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.819+0000 7f0efffff640 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.820+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d3ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c6ed0 tx=0x7f0ef80ec040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0962257df2f018 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.820+0000 7f0efd7fa640 1 -- 172.21.15.50:0/1751719379 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8049040 con 0x5579814092f0 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.820+0000 7f0efd7fa640 1 -- 172.21.15.50:0/1751719379 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef80cdf60 con 0x5579814092f0 2024-05-17T04:38:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.820+0000 7f0efd7fa640 1 -- 172.21.15.50:0/1751719379 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80eae60 con 0x5579814092f0 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- 172.21.15.50:0/1751719379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 msgr2=0x5579817d3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d3ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c6ed0 tx=0x7f0ef80ec040 comp rx=0 tx=0).stop 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- 172.21.15.50:0/1751719379 shutdown_connections 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814321b0 0x5579817d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 --2- 172.21.15.50:0/1751719379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814092f0 0x5579817d3ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- 172.21.15.50:0/1751719379 >> 172.21.15.50:0/1751719379 conn(0x557981426300 msgr2=0x5579817d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- 172.21.15.50:0/1751719379 shutdown_connections 2024-05-17T04:38:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- 172.21.15.50:0/1751719379 wait complete. 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.821+0000 7f0f15073740 1 -- start start 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 Processor -- start 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 0x5579817e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x557981852d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ac790 con 0x5579814114d0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081adc00 con 0x5579814321b0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081ace60 con 0x5579814092f0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f15073740 10 client.0 _ll_register_callbacks cb 0x7f0f00011e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 0x5579817e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0efffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57444/0 (socket says 172.21.15.50:57444) 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0efffff640 1 -- 172.21.15.50:0/1068936949 learned_addr learned my addr 172.21.15.50:0/1068936949 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:38:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f07fff640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x557981852d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 -- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 msgr2=0x557981852d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x557981852d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 -- 172.21.15.50:0/1068936949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 msgr2=0x5579817e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0f077fe640 1 -- 172.21.15.50:0/1068936949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579817e7010 con 0x5579814092f0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.822+0000 7f0efffff640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0f077fe640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 0x5579817e3ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c6ed0 tx=0x7f0ef804b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c60cf23fdfd143b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8049070 con 0x5579814092f0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef80cdf60 con 0x5579814092f0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0ef80c6250 con 0x5579814092f0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ef80eae60 con 0x5579814092f0 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0f15073740 10 client.25258 fetch_fsmap learned FSMap version 31 2024-05-17T04:38:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557981854270 con 0x5579814092f0 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef80cdf60 con 0x5579814092f0 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.823+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0ef80c55d0 con 0x5579814092f0 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 10 client.25258 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 20 client.25258 populate_metadata read hostname 'smithi050' 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813c61c0 0x557981856be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 4 client.25258 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579817433f0 con 0x5579813c61c0 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0efffff640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813c61c0 0x557981856be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0ee6ffd640 10 client.25258 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.824+0000 7f0efffff640 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813c61c0 0x557981856be0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566b262d4f83b0e5 server_cookie=5035463872568eb4 in_seq=0 out_seq=0 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0ee6ffd640 1 -- 172.21.15.50:0/1068936949 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5579817433f0 con 0x5579813c61c0 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0ee6ffd640 10 client.25258 handle_command_reply: tid=0 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 client.25258 shutdown 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 msgr2=0x5579817e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 0x5579817e3ae0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c6ed0 tx=0x7f0ef804b000 comp rx=0 tx=0).stop 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 shutdown_connections 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579814321b0 0x557981852d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.841+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5579813c61c0 0x557981856be0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579814114d0 0x5579817e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 1 --2- 172.21.15.50:0/1068936949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5579814092f0 0x5579817e3ae0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0ee6ffd640 0 client.25258 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 >> 172.21.15.50:0/1068936949 conn(0x557981426300 msgr2=0x5579817cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 shutdown_connections 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 1 -- 172.21.15.50:0/1068936949 wait complete. 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f15073740 20 client.25258 trim_cache size 0 max 16384 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:38:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:38:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 msgr2=0x7f0ef40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 0x7f0ef40b4cf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5579817d6960 tx=0x7f0f00009500 comp rx=0 tx=0).stop 2024-05-17T04:38:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 msgr2=0x7f0f081034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:38:58.845 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8031460 tx=0x7f0ef8005e70 comp rx=0 tx=0).stop 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 shutdown_connections 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f08108f40 0x7f0f081020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f0ef40b27e0 0x7f0ef40b4cf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f081065e0 0x7f0f08101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 --2- 172.21.15.50:0/2373773769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f08103c80 0x7f0f081034f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 >> 172.21.15.50:0/2373773769 conn(0x7f0f080fd8b0 msgr2=0x7f0f081049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 shutdown_connections 2024-05-17T04:38:58.846 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:38:58.842+0000 7f0f0e1bd640 1 -- 172.21.15.50:0/2373773769 wait complete. 2024-05-17T04:39:00.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:00 smithi105 ceph-mon[34795]: pgmap v519: 129 pgs: 129 active+clean; 939 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 37 op/s 2024-05-17T04:39:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:00 smithi110 ceph-mon[35497]: pgmap v519: 129 pgs: 129 active+clean; 939 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 37 op/s 2024-05-17T04:39:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:00 smithi050 ceph-mon[31739]: pgmap v519: 129 pgs: 129 active+clean; 939 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 37 op/s 2024-05-17T04:39:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:02 smithi105 ceph-mon[34795]: pgmap v520: 129 pgs: 129 active+clean; 908 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 70 op/s 2024-05-17T04:39:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:02 smithi110 ceph-mon[35497]: pgmap v520: 129 pgs: 129 active+clean; 908 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 70 op/s 2024-05-17T04:39:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:02 smithi050 ceph-mon[31739]: pgmap v520: 129 pgs: 129 active+clean; 908 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 25 MiB/s wr, 70 op/s 2024-05-17T04:39:02.800 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 5.0 s, 157.5 tps, lat 71.382 ms stddev 169.919 2024-05-17T04:39:04.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:04 smithi105 ceph-mon[34795]: pgmap v521: 129 pgs: 129 active+clean; 909 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 91 op/s 2024-05-17T04:39:04.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:04 smithi110 ceph-mon[35497]: pgmap v521: 129 pgs: 129 active+clean; 909 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 91 op/s 2024-05-17T04:39:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:04 smithi050 ceph-mon[31739]: pgmap v521: 129 pgs: 129 active+clean; 909 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 22 MiB/s wr, 91 op/s 2024-05-17T04:39:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:06 smithi105 ceph-mon[34795]: pgmap v522: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 220 op/s 2024-05-17T04:39:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:06 smithi110 ceph-mon[35497]: pgmap v522: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 220 op/s 2024-05-17T04:39:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:06 smithi050 ceph-mon[31739]: pgmap v522: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 16 MiB/s wr, 220 op/s 2024-05-17T04:39:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 10.0 s, 242.0 tps, lat 745.498 ms stddev 2231.842 2024-05-17T04:39:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:08 smithi105 ceph-mon[34795]: pgmap v523: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 211 op/s 2024-05-17T04:39:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:08 smithi110 ceph-mon[35497]: pgmap v523: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 211 op/s 2024-05-17T04:39:08.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:08 smithi050 ceph-mon[31739]: pgmap v523: 129 pgs: 129 active+clean; 911 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.8 MiB/s wr, 211 op/s 2024-05-17T04:39:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:10 smithi105 ceph-mon[34795]: pgmap v524: 129 pgs: 129 active+clean; 924 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.7 MiB/s wr, 225 op/s 2024-05-17T04:39:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:10 smithi110 ceph-mon[35497]: pgmap v524: 129 pgs: 129 active+clean; 924 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.7 MiB/s wr, 225 op/s 2024-05-17T04:39:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:10 smithi050 ceph-mon[31739]: pgmap v524: 129 pgs: 129 active+clean; 924 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.7 MiB/s wr, 225 op/s 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 -- 172.21.15.50:0/2838074541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31641064b0 msgr2=0x7f31641068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 --2- 172.21.15.50:0/2838074541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31641064b0 0x7f31641068c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3158009900 tx=0x7f315802a410 comp rx=0 tx=0).stop 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 -- 172.21.15.50:0/2838074541 shutdown_connections 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 --2- 172.21.15.50:0/2838074541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3164106e00 0x7f316410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 --2- 172.21.15.50:0/2838074541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31641064b0 0x7f31641068c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 --2- 172.21.15.50:0/2838074541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f31641057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 -- 172.21.15.50:0/2838074541 >> 172.21.15.50:0/2838074541 conn(0x7f3164100b70 msgr2=0x7f3164103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 -- 172.21.15.50:0/2838074541 shutdown_connections 2024-05-17T04:39:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.975+0000 7f3169f4a640 1 -- 172.21.15.50:0/2838074541 wait complete. 2024-05-17T04:39:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.976+0000 7f3169f4a640 1 Processor -- start 2024-05-17T04:39:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.976+0000 7f3169f4a640 1 -- start start 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.976+0000 7f3169f4a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.976+0000 7f3169f4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31641064b0 0x7f31641a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3169f4a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3164106e00 0x7f31641a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3169f4a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3164110d70 con 0x7f3164106e00 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3169f4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3164110bf0 con 0x7f31641064b0 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3169f4a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3164110ef0 con 0x7f3164105390 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48658/0 (socket says 172.21.15.50:48658) 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 -- 172.21.15.50:0/1501112709 learned_addr learned my addr 172.21.15.50:0/1501112709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3162ffd640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31641064b0 0x7f31641a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f3163fff640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3164106e00 0x7f31641a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 -- 172.21.15.50:0/1501112709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31641064b0 msgr2=0x7f31641a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31641064b0 0x7f31641a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 -- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3164106e00 msgr2=0x7f31641a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3164106e00 0x7f31641a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 -- 172.21.15.50:0/1501112709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641a7930 con 0x7f3164105390 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.977+0000 7f31637fe640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9c0 tx=0x7f315400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.978+0000 7f3160ff9640 1 -- 172.21.15.50:0/1501112709 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315400cd80 con 0x7f3164105390 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.978+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31641a7c20 con 0x7f3164105390 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.978+0000 7f3160ff9640 1 -- 172.21.15.50:0/1501112709 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31540054b0 con 0x7f3164105390 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.978+0000 7f3160ff9640 1 -- 172.21.15.50:0/1501112709 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154010430 con 0x7f3164105390 2024-05-17T04:39:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.978+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641a8170 con 0x7f3164105390 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 Processor -- start 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 -- start start 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x5596833f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x5596833ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55968336acc0 con 0x5596830344d0 2024-05-17T04:39:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55968335a860 con 0x55968302c2f0 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.981+0000 7f3170e08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55968335e8a0 con 0x5596830551b0 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x5596833ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48670/0 (socket says 172.21.15.50:48670) 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 -- 172.21.15.50:0/2831693419 learned_addr learned my addr 172.21.15.50:0/2831693419 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 -- 172.21.15.50:0/2831693419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 msgr2=0x5596833f8cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x5596833f8cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 -- 172.21.15.50:0/2831693419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 msgr2=0x5596833ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x5596833ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 -- 172.21.15.50:0/2831693419 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559683322d50 con 0x5596830551b0 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3160ff9640 1 -- 172.21.15.50:0/1501112709 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f31540105d0 con 0x7f3164105390 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3162ffd640 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x5596833ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3163fff640 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f315c009fd0 tx=0x7f315c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fc1b6ac6e91e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.982+0000 7f3149ffb640 1 -- 172.21.15.50:0/2831693419 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c004d20 con 0x5596830551b0 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.983+0000 7f3149ffb640 1 -- 172.21.15.50:0/2831693419 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315c004050 con 0x5596830551b0 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.983+0000 7f3149ffb640 1 -- 172.21.15.50:0/2831693419 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c005260 con 0x5596830551b0 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.983+0000 7f3160ff9640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 0x7f314c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.983+0000 7f31637fe640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 0x7f314c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3160ff9640 1 -- 172.21.15.50:0/1501112709 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f31540e2b50 con 0x7f3164105390 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 -- 172.21.15.50:0/2831693419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 msgr2=0x5596833f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f315c009fd0 tx=0x7f315c0118d0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 -- 172.21.15.50:0/2831693419 shutdown_connections 2024-05-17T04:39:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x5596833ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x5596833f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 --2- 172.21.15.50:0/2831693419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830551b0 0x5596833f8770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 -- 172.21.15.50:0/2831693419 >> 172.21.15.50:0/2831693419 conn(0x559683049300 msgr2=0x5596833f41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 -- 172.21.15.50:0/2831693419 shutdown_connections 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f31637fe640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 0x7f314c0b4cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31540a9ae0 tx=0x7f3154013d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.984+0000 7f3170e08740 1 -- 172.21.15.50:0/2831693419 wait complete. 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 -- start start 2024-05-17T04:39:11.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 Processor -- start 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 0x559683402580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559683296ab0 con 0x5596830551b0 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596833174d0 con 0x5596830344d0 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.985+0000 7f3170e08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559683400690 con 0x55968302c2f0 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3170e08740 10 client.0 _ll_register_callbacks cb 0x559683355950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48678/0 (socket says 172.21.15.50:48678) 2024-05-17T04:39:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 -- 172.21.15.50:0/4100387423 learned_addr learned my addr 172.21.15.50:0/4100387423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3162ffd640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 0x559683402580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 -- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 msgr2=0x559683402580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 0x559683402580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 -- 172.21.15.50:0/4100387423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 msgr2=0x5596834094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3163fff640 1 -- 172.21.15.50:0/4100387423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559683318340 con 0x55968302c2f0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.986+0000 7f3162ffd640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 0x559683402580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3163fff640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f315c004760 tx=0x7f315c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d5b150e9826eef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c009c70 con 0x55968302c2f0 2024-05-17T04:39:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315c004050 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f315c00f4e0 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f315c036050 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3170e08740 10 client.25270 fetch_fsmap learned FSMap version 31 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.987+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5596830c0790 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315c004050 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f315c025040 con 0x55968302c2f0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 10 client.25270 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 10 client.25270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 20 client.25270 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559683013d80 0x55968340fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 4 client.25270 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968336cc10 con 0x559683013d80 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.988+0000 7f3162ffd640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559683013d80 0x55968340fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.989+0000 7f3142ffd640 10 client.25270 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.989+0000 7f3162ffd640 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559683013d80 0x55968340fb70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d620262e58f2279 server_cookie=bbd22f3858ee5dae in_seq=0 out_seq=0 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3142ffd640 1 -- 172.21.15.50:0/4100387423 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55968336cc10 con 0x559683013d80 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3142ffd640 10 client.25270 handle_command_reply: tid=0 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 client.25270 shutdown 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 msgr2=0x559683402040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f315c004760 tx=0x7f315c000f40 comp rx=0 tx=0).stop 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 shutdown_connections 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559683013d80 0x55968340fb70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830344d0 0x559683402580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 --2- 172.21.15.50:0/4100387423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x559683402040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 >> 172.21.15.50:0/4100387423 conn(0x559683049300 msgr2=0x5596833f31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3142ffd640 0 client.25270 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 shutdown_connections 2024-05-17T04:39:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.990+0000 7f3170e08740 1 -- 172.21.15.50:0/4100387423 wait complete. 2024-05-17T04:39:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:11.991+0000 7f3170e08740 20 client.25270 trim_cache size 0 max 16384 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.014+0000 7f3170e08740 1 Processor -- start 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 -- start start 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x559683409400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830551b0 0x559683409940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b5000 con 0x5596830344d0 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b2870 con 0x5596830551b0 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3170e08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b06c0 con 0x55968302c2f0 2024-05-17T04:39:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x559683409400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x559683409400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44256/0 (socket says 172.21.15.50:44256) 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3162ffd640 1 -- 172.21.15.50:0/1173164234 learned_addr learned my addr 172.21.15.50:0/1173164234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3163fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48686/0 (socket says 172.21.15.50:48686) 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.015+0000 7f3163fff640 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830551b0 0x559683409940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 -- 172.21.15.50:0/1173164234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830551b0 msgr2=0x559683409940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830551b0 0x559683409940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 -- 172.21.15.50:0/1173164234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 msgr2=0x559683409400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x559683409400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 -- 172.21.15.50:0/1173164234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559683139620 con 0x55968302c2f0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f3163fff640 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f315c009560 tx=0x7f315c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdb4772c2f5f240 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f31497fa640 1 -- 172.21.15.50:0/1173164234 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c009b30 con 0x55968302c2f0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f31497fa640 1 -- 172.21.15.50:0/1173164234 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315c02d070 con 0x55968302c2f0 2024-05-17T04:39:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.016+0000 7f31497fa640 1 -- 172.21.15.50:0/1173164234 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c00f4e0 con 0x55968302c2f0 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 -- 172.21.15.50:0/1173164234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 msgr2=0x5596833f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f315c009560 tx=0x7f315c0118d0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 -- 172.21.15.50:0/1173164234 shutdown_connections 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596830551b0 0x559683409940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830344d0 0x559683409400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 --2- 172.21.15.50:0/1173164234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55968302c2f0 0x5596833f2ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 -- 172.21.15.50:0/1173164234 >> 172.21.15.50:0/1173164234 conn(0x559683049300 msgr2=0x5596833f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 -- 172.21.15.50:0/1173164234 shutdown_connections 2024-05-17T04:39:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.017+0000 7f3170e08740 1 -- 172.21.15.50:0/1173164234 wait complete. 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 -- start start 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 Processor -- start 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x559683404480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b0d30 con 0x5596830551b0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b21a0 con 0x55968302c2f0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.018+0000 7f3170e08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f31641b1400 con 0x5596830344d0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3170e08740 10 client.0 _ll_register_callbacks cb 0x7f3154005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x559683404480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3162ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48700/0 (socket says 172.21.15.50:48700) 2024-05-17T04:39:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3162ffd640 1 -- 172.21.15.50:0/1042876949 learned_addr learned my addr 172.21.15.50:0/1042876949 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 -- 172.21.15.50:0/1042876949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 msgr2=0x5596834049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 -- 172.21.15.50:0/1042876949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 msgr2=0x5596834098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 -- 172.21.15.50:0/1042876949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559683409eb0 con 0x55968302c2f0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3162ffd640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3163fff640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x559683404480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f315c009560 tx=0x7f315c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c3c3deb5d4d83d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.019+0000 7f3142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c041070 con 0x55968302c2f0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315c02d070 con 0x55968302c2f0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f315c009a50 con 0x55968302c2f0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f315c051050 con 0x55968302c2f0 2024-05-17T04:39:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3170e08740 10 client.25280 fetch_fsmap learned FSMap version 31 2024-05-17T04:39:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559683476d50 con 0x55968302c2f0 2024-05-17T04:39:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f315c02d070 con 0x55968302c2f0 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.020+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f315c00f770 con 0x55968302c2f0 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 10 client.25280 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 20 client.25280 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559682fe91c0 0x559683479630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 4 client.25280 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596833663f0 con 0x559682fe91c0 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3162ffd640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559682fe91c0 0x559683479630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3142ffd640 10 client.25280 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.021+0000 7f3162ffd640 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559682fe91c0 0x559683479630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39bd3034808e100 server_cookie=6f1157d3faf69974 in_seq=0 out_seq=0 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3142ffd640 1 -- 172.21.15.50:0/1042876949 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5596833663f0 con 0x559682fe91c0 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3142ffd640 10 client.25280 handle_command_reply: tid=0 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3170e08740 1 client.25280 shutdown 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 msgr2=0x559683404480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x559683404480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f315c009560 tx=0x7f315c0096c0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.037+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 shutdown_connections 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559682fe91c0 0x559683479630 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5596830551b0 0x5596834098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3142ffd640 0 client.25280 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596830344d0 0x5596834049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 --2- 172.21.15.50:0/1042876949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55968302c2f0 0x559683404480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 >> 172.21.15.50:0/1042876949 conn(0x559683049300 msgr2=0x5596833f6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 shutdown_connections 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 1 -- 172.21.15.50:0/1042876949 wait complete. 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.038+0000 7f3170e08740 20 client.25280 trim_cache size 0 max 16384 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:39:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 msgr2=0x7f314c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 0x7f314c0b4cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31540a9ae0 tx=0x7f3154013d50 comp rx=0 tx=0).stop 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 msgr2=0x7f316419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9c0 tx=0x7f315400ee80 comp rx=0 tx=0).stop 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 shutdown_connections 2024-05-17T04:39:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f314c0b27e0 0x7f314c0b4cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3164106e00 0x7f31641a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31641064b0 0x7f31641a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 --2- 172.21.15.50:0/1501112709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3164105390 0x7f316419fdf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 >> 172.21.15.50:0/1501112709 conn(0x7f3164100b70 msgr2=0x7f3164079190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 shutdown_connections 2024-05-17T04:39:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:12.039+0000 7f3169f4a640 1 -- 172.21.15.50:0/1501112709 wait complete. 2024-05-17T04:39:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:12 smithi105 ceph-mon[34795]: pgmap v525: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-05-17T04:39:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:12 smithi110 ceph-mon[35497]: pgmap v525: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-05-17T04:39:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:12 smithi050 ceph-mon[31739]: pgmap v525: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-05-17T04:39:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 15.0 s, 324.6 tps, lat 305.899 ms stddev 22.607 2024-05-17T04:39:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:14 smithi105 ceph-mon[34795]: pgmap v526: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T04:39:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:14 smithi110 ceph-mon[35497]: pgmap v526: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T04:39:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:14 smithi050 ceph-mon[31739]: pgmap v526: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-05-17T04:39:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:16 smithi105 ceph-mon[34795]: pgmap v527: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-05-17T04:39:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:16 smithi110 ceph-mon[35497]: pgmap v527: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-05-17T04:39:16.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:16 smithi050 ceph-mon[31739]: pgmap v527: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.2 MiB/s wr, 385 op/s 2024-05-17T04:39:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 20.0 s, 323.2 tps, lat 308.800 ms stddev 25.819 2024-05-17T04:39:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:18 smithi105 ceph-mon[34795]: pgmap v528: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-05-17T04:39:18.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:18 smithi110 ceph-mon[35497]: pgmap v528: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-05-17T04:39:18.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:18 smithi050 ceph-mon[31739]: pgmap v528: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-05-17T04:39:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:20 smithi105 ceph-mon[34795]: pgmap v529: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-17T04:39:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:20 smithi110 ceph-mon[35497]: pgmap v529: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-17T04:39:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:20 smithi050 ceph-mon[31739]: pgmap v529: 129 pgs: 129 active+clean; 929 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-05-17T04:39:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:22 smithi105 ceph-mon[34795]: pgmap v530: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-17T04:39:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:22 smithi110 ceph-mon[35497]: pgmap v530: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-17T04:39:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:22 smithi050 ceph-mon[31739]: pgmap v530: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 344 op/s 2024-05-17T04:39:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 25.0 s, 326.4 tps, lat 302.978 ms stddev 29.702 2024-05-17T04:39:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:24 smithi105 ceph-mon[34795]: pgmap v531: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-05-17T04:39:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:24 smithi110 ceph-mon[35497]: pgmap v531: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-05-17T04:39:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:24 smithi050 ceph-mon[31739]: pgmap v531: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 -- 172.21.15.50:0/94102962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284104c00 msgr2=0x7fe28410b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/94102962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284104c00 0x7fe28410b500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009ca0 tx=0x7fe27c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 -- 172.21.15.50:0/94102962 shutdown_connections 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/94102962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284104c00 0x7fe28410b500 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/94102962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 0x7fe2841046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/94102962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284103190 0x7fe2841035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 -- 172.21.15.50:0/94102962 >> 172.21.15.50:0/94102962 conn(0x7fe2840fe9b0 msgr2=0x7fe284100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 -- 172.21.15.50:0/94102962 shutdown_connections 2024-05-17T04:39:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.172+0000 7fe28aa3d640 1 -- 172.21.15.50:0/94102962 wait complete. 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 Processor -- start 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 -- start start 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 0x7fe2841997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 0x7fe284199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 0x7fe2841a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410eb20 con 0x7fe284103190 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410e9a0 con 0x7fe2841042b0 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.173+0000 7fe28aa3d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410eca0 con 0x7fe284104c00 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe288fda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 0x7fe2841a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe288fda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 0x7fe2841a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:36988/0 (socket says 172.21.15.50:36988) 2024-05-17T04:39:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe288fda640 1 -- 172.21.15.50:0/3028609778 learned_addr learned my addr 172.21.15.50:0/3028609778 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 0x7fe2841997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 0x7fe284199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 -- 172.21.15.50:0/3028609778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 msgr2=0x7fe2841a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 0x7fe2841a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 -- 172.21.15.50:0/3028609778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 msgr2=0x7fe284199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 0x7fe284199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 -- 172.21.15.50:0/3028609778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2841a1300 con 0x7fe284103190 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.174+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 0x7fe2841997c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e9c0 tx=0x7fe27400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.175+0000 7fe2817fa640 1 -- 172.21.15.50:0/3028609778 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27400ccc0 con 0x7fe284103190 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.175+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2841a15f0 con 0x7fe284103190 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.175+0000 7fe2817fa640 1 -- 172.21.15.50:0/3028609778 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe274005020 con 0x7fe284103190 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.175+0000 7fe2817fa640 1 -- 172.21.15.50:0/3028609778 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27400ce60 con 0x7fe284103190 2024-05-17T04:39:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.175+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2841a1af0 con 0x7fe284103190 2024-05-17T04:39:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.178+0000 7fe2919d9740 1 Processor -- start 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 -- start start 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 0x559782953bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782956350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2817fa640 1 -- 172.21.15.50:0/3028609778 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe274004a20 con 0x7fe284103190 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5597828b8cc0 con 0x55978257a2f0 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597828a8860 con 0x5597825a31b0 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2919d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5597828ac8a0 con 0x5597825824d0 2024-05-17T04:39:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42336/0 (socket says 172.21.15.50:42336) 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 -- 172.21.15.50:0/3532585605 learned_addr learned my addr 172.21.15.50:0/3532585605 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe283fff640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 0x559782953bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782956350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 -- 172.21.15.50:0/3532585605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 msgr2=0x559782956350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782956350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 -- 172.21.15.50:0/3532585605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 msgr2=0x559782953bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 0x559782953bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.179+0000 7fe288fda640 1 -- 172.21.15.50:0/3532585605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782870d50 con 0x5597825a31b0 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe283fff640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 0x559782953bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782956350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe288fda640 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004af0 tx=0x7fe27c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aaa0698f686f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe2817fa640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 0x7fe26c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 0x7fe26c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe26a7fc640 1 -- 172.21.15.50:0/3532585605 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031650 con 0x5597825a31b0 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.180+0000 7fe26a7fc640 1 -- 172.21.15.50:0/3532585605 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe27c036040 con 0x5597825a31b0 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.181+0000 7fe26a7fc640 1 -- 172.21.15.50:0/3532585605 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0043c0 con 0x5597825a31b0 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.181+0000 7fe2817fa640 1 -- 172.21.15.50:0/3028609778 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe2740e24d0 con 0x7fe284103190 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- 172.21.15.50:0/3532585605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 msgr2=0x559782951670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004af0 tx=0x7fe27c009870 comp rx=0 tx=0).stop 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe283fff640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 0x7fe26c0b4cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559782945ac0 tx=0x7fe2740e4040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- 172.21.15.50:0/3532585605 shutdown_connections 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782956350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55978257a2f0 0x559782953bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3532585605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782951670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- 172.21.15.50:0/3532585605 >> 172.21.15.50:0/3532585605 conn(0x559782597300 msgr2=0x559782942120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- 172.21.15.50:0/3532585605 shutdown_connections 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- 172.21.15.50:0/3532585605 wait complete. 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.182+0000 7fe2919d9740 1 -- start start 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 Processor -- start 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x559782970ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5597827e4ab0 con 0x5597825a31b0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597828654d0 con 0x55978257a2f0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559782967ca0 con 0x5597825824d0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2919d9740 10 client.0 _ll_register_callbacks cb 0x5597828a3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe288fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe288fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42352/0 (socket says 172.21.15.50:42352) 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe2837fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37012/0 (socket says 172.21.15.50:37012) 2024-05-17T04:39:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe288fda640 1 -- 172.21.15.50:0/2630968316 learned_addr learned my addr 172.21.15.50:0/2630968316 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.183+0000 7fe288fda640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x559782970ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 -- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 msgr2=0x559782969700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 -- 172.21.15.50:0/2630968316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 msgr2=0x559782970ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x559782970ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 -- 172.21.15.50:0/2630968316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559782866340 con 0x5597825824d0 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe288fda640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x559782942e10 tx=0x7fe278013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e44d81efabafa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.184+0000 7fe2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:39:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278004460 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe278011990 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27c036040 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe278011b90 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2919d9740 10 client.25285 fetch_fsmap learned FSMap version 31 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55978260e790 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe278011d30 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe27801a070 con 0x5597825824d0 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2919d9740 10 client.25285 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.185+0000 7fe2919d9740 10 client.25285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe2919d9740 20 client.25285 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559782561d80 0x559782976f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe2919d9740 4 client.25285 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe27c036040 con 0x559782561d80 2024-05-17T04:39:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe288fda640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559782561d80 0x559782976f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe2637fe640 10 client.25285 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.186+0000 7fe288fda640 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559782561d80 0x559782976f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9acd1c87b69555 server_cookie=9af97576733a0243 in_seq=0 out_seq=0 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.187+0000 7fe2637fe640 1 -- 172.21.15.50:0/2630968316 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe27c036040 con 0x559782561d80 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.187+0000 7fe2637fe640 10 client.25285 handle_command_reply: tid=0 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.187+0000 7fe2919d9740 1 client.25285 shutdown 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 msgr2=0x559782969c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x559782942e10 tx=0x7fe278013b70 comp rx=0 tx=0).stop 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 shutdown_connections 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559782561d80 0x559782976f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x559782970ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5597825824d0 0x559782969c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2630968316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978257a2f0 0x559782969700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2637fe640 0 client.25285 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 >> 172.21.15.50:0/2630968316 conn(0x559782597300 msgr2=0x5597829411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 shutdown_connections 2024-05-17T04:39:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 1 -- 172.21.15.50:0/2630968316 wait complete. 2024-05-17T04:39:25.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.188+0000 7fe2919d9740 20 client.25285 trim_cache size 0 max 16384 2024-05-17T04:39:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 Processor -- start 2024-05-17T04:39:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 -- start start 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x55978297b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782971000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841a3ed0 con 0x5597825824d0 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe284076b10 con 0x5597825a31b0 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.211+0000 7fe2919d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe284074960 con 0x55978257a2f0 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe288fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782971000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38540/0 (socket says 172.21.15.50:38540) 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 -- 172.21.15.50:0/3928366138 learned_addr learned my addr 172.21.15.50:0/3928366138 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe288fda640 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x55978297b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 -- 172.21.15.50:0/3928366138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 msgr2=0x55978297b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x55978297b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 -- 172.21.15.50:0/3928366138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 msgr2=0x559782971000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782971000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 -- 172.21.15.50:0/3928366138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782687620 con 0x5597825824d0 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.212+0000 7fe2837fe640 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe27800f6f0 tx=0x7fe278011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac56c4a837c2c9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.213+0000 7fe269ffb640 1 -- 172.21.15.50:0/3928366138 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278011d30 con 0x5597825824d0 2024-05-17T04:39:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.213+0000 7fe269ffb640 1 -- 172.21.15.50:0/3928366138 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe27802e070 con 0x5597825824d0 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.213+0000 7fe269ffb640 1 -- 172.21.15.50:0/3928366138 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278003850 con 0x5597825824d0 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 -- 172.21.15.50:0/3928366138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 msgr2=0x559782970ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe27800f6f0 tx=0x7fe278011b20 comp rx=0 tx=0).stop 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 -- 172.21.15.50:0/3928366138 shutdown_connections 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825a31b0 0x559782971000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825824d0 0x559782970ac0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 --2- 172.21.15.50:0/3928366138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x55978297b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 -- 172.21.15.50:0/3928366138 >> 172.21.15.50:0/3928366138 conn(0x559782597300 msgr2=0x559782950660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 -- 172.21.15.50:0/3928366138 shutdown_connections 2024-05-17T04:39:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.214+0000 7fe2919d9740 1 -- 172.21.15.50:0/3928366138 wait complete. 2024-05-17T04:39:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 -- start start 2024-05-17T04:39:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 Processor -- start 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825824d0 0x5597829709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x5597829dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe284074fd0 con 0x5597825a31b0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe284076440 con 0x5597825824d0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2840756a0 con 0x55978257a2f0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe2919d9740 10 client.0 _ll_register_callbacks cb 0x7fe2780097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.215+0000 7fe288fda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37028/0 (socket says 172.21.15.50:37028) 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 -- 172.21.15.50:0/2296666178 learned_addr learned my addr 172.21.15.50:0/2296666178 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x5597829dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825824d0 0x5597829709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 -- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825824d0 msgr2=0x5597829709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825824d0 0x5597829709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 -- 172.21.15.50:0/2296666178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 msgr2=0x5597829dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x5597829dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 -- 172.21.15.50:0/2296666178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597829dd870 con 0x55978257a2f0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe288fda640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004af0 tx=0x7fe278009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d71b72513a8e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.216+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c05e070 con 0x55978257a2f0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27802e070 con 0x55978257a2f0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe27c004000 con 0x55978257a2f0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe27c06f050 con 0x55978257a2f0 2024-05-17T04:39:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 10 client.25291 fetch_fsmap learned FSMap version 31 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5597829deb60 con 0x55978257a2f0 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe27802e070 con 0x55978257a2f0 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe27c064040 con 0x55978257a2f0 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 10 client.25291 fetch_fsmap finished waiting for FSMap version 31 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 20 client.25291 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.217+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5597825371c0 0x5597829e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.218+0000 7fe2919d9740 4 client.25291 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.218+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597828bac10 con 0x5597825371c0 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.218+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5597825371c0 0x5597829e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.218+0000 7fe2637fe640 10 client.25291 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.218+0000 7fe2837fe640 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5597825371c0 0x5597829e1250 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1183b5cf76430ab server_cookie=67b2066f12c25bb2 in_seq=0 out_seq=0 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.234+0000 7fe2637fe640 1 -- 172.21.15.50:0/2296666178 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5597828bac10 con 0x5597825371c0 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.234+0000 7fe2637fe640 10 client.25291 handle_command_reply: tid=0 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.234+0000 7fe2919d9740 1 client.25291 shutdown 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 msgr2=0x559782970470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004af0 tx=0x7fe278009d80 comp rx=0 tx=0).stop 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 shutdown_connections 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5597825371c0 0x5597829e1250 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5597825a31b0 0x5597829dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597825824d0 0x5597829709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 --2- 172.21.15.50:0/2296666178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55978257a2f0 0x559782970470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 >> 172.21.15.50:0/2296666178 conn(0x559782597300 msgr2=0x5597829407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2637fe640 0 client.25291 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 shutdown_connections 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 1 -- 172.21.15.50:0/2296666178 wait complete. 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.235+0000 7fe2919d9740 20 client.25291 trim_cache size 0 max 16384 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:39:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 msgr2=0x7fe26c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 0x7fe26c0b4cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559782945ac0 tx=0x7fe2740e4040 comp rx=0 tx=0).stop 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 msgr2=0x7fe2841997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 0x7fe2841997c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e9c0 tx=0x7fe27400ee80 comp rx=0 tx=0).stop 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 shutdown_connections 2024-05-17T04:39:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe284104c00 0x7fe2841a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2841042b0 0x7fe284199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe26c0b27e0 0x7fe26c0b4cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 --2- 172.21.15.50:0/3028609778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe284103190 0x7fe2841997c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 >> 172.21.15.50:0/3028609778 conn(0x7fe2840fe9b0 msgr2=0x7fe284107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 shutdown_connections 2024-05-17T04:39:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:25.236+0000 7fe28aa3d640 1 -- 172.21.15.50:0/3028609778 wait complete. 2024-05-17T04:39:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:26 smithi105 ceph-mon[34795]: pgmap v532: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-05-17T04:39:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:26 smithi110 ceph-mon[35497]: pgmap v532: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-05-17T04:39:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:26 smithi050 ceph-mon[31739]: pgmap v532: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 323 op/s 2024-05-17T04:39:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 30.0 s, 323.0 tps, lat 308.706 ms stddev 48.150 2024-05-17T04:39:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:28 smithi105 ceph-mon[34795]: pgmap v533: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-05-17T04:39:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:28 smithi110 ceph-mon[35497]: pgmap v533: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-05-17T04:39:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:28 smithi050 ceph-mon[31739]: pgmap v533: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-05-17T04:39:29.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:29 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:29.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:39:29.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:29 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:29.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:39:29.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:29 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:29.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:39:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:30 smithi105 ceph-mon[34795]: pgmap v534: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-05-17T04:39:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:39:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:39:30.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:39:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:30 smithi110 ceph-mon[35497]: pgmap v534: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-05-17T04:39:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:39:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:39:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:39:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:30 smithi050 ceph-mon[31739]: pgmap v534: 129 pgs: 129 active+clean; 929 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-05-17T04:39:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:39:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:39:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:39:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:32 smithi105 ceph-mon[34795]: pgmap v535: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-05-17T04:39:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:32 smithi110 ceph-mon[35497]: pgmap v535: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-05-17T04:39:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:32 smithi050 ceph-mon[31739]: pgmap v535: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-05-17T04:39:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 35.0 s, 138.0 tps, lat 704.816 ms stddev 230.614 2024-05-17T04:39:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:34 smithi105 ceph-mon[34795]: pgmap v536: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:39:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:34 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:34 smithi110 ceph-mon[35497]: pgmap v536: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:39:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:34 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:34 smithi050 ceph-mon[31739]: pgmap v536: 129 pgs: 129 active+clean; 929 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:39:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:34 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:39:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:36 smithi105 ceph-mon[34795]: pgmap v537: 129 pgs: 129 active+clean; 930 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.2 MiB/s wr, 317 op/s 2024-05-17T04:39:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:36 smithi110 ceph-mon[35497]: pgmap v537: 129 pgs: 129 active+clean; 930 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.2 MiB/s wr, 317 op/s 2024-05-17T04:39:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:36 smithi050 ceph-mon[31739]: pgmap v537: 129 pgs: 129 active+clean; 930 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.2 MiB/s wr, 317 op/s 2024-05-17T04:39:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 40.0 s, 114.2 tps, lat 733.370 ms stddev 177.322 2024-05-17T04:39:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 -- 172.21.15.50:0/1925719262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470103ca0 msgr2=0x7f44701060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1925719262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470103ca0 0x7f44701060c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4464009900 tx=0x7f446402a410 comp rx=0 tx=0).stop 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 -- 172.21.15.50:0/1925719262 shutdown_connections 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1925719262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 0x7f447010b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1925719262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470106600 0x7f4470108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1925719262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470103ca0 0x7f44701060c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 -- 172.21.15.50:0/1925719262 >> 172.21.15.50:0/1925719262 conn(0x7f44700fd8b0 msgr2=0x7f44700ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 -- 172.21.15.50:0/1925719262 shutdown_connections 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.349+0000 7f4476e3c640 1 -- 172.21.15.50:0/1925719262 wait complete. 2024-05-17T04:39:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 Processor -- start 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 -- start start 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 0x7f4470199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 0x7f44701a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f447010eb80 con 0x7f4470106600 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f447010ea00 con 0x7f4470108f60 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4476e3c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f447010ed00 con 0x7f4470103ca0 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f446ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 0x7f4470199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.350+0000 7f446ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39884/0 (socket says 172.21.15.50:39884) 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 -- 172.21.15.50:0/1974544495 learned_addr learned my addr 172.21.15.50:0/1974544495 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f44753d9640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 0x7f44701a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 -- 172.21.15.50:0/1974544495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 msgr2=0x7f4470199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 0x7f4470199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 -- 172.21.15.50:0/1974544495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 msgr2=0x7f44701a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 0x7f44701a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 -- 172.21.15.50:0/1974544495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44701a1340 con 0x7f4470106600 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f4474bd8640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 0x7f4470199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446ffff640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f445c00e9f0 tx=0x7f445c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446dffb640 1 -- 172.21.15.50:0/1974544495 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c00cd80 con 0x7f4470106600 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44701a1630 con 0x7f4470106600 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446dffb640 1 -- 172.21.15.50:0/1974544495 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f445c0054b0 con 0x7f4470106600 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.351+0000 7f446dffb640 1 -- 172.21.15.50:0/1974544495 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c010430 con 0x7f4470106600 2024-05-17T04:39:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.352+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44701a1b80 con 0x7f4470106600 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.354+0000 7f447ddcc740 1 Processor -- start 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.354+0000 7f447ddcc740 1 -- start start 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8912f0 0x558c6fc5dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c6fbcfcc0 con 0x558c6f8912f0 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c6fbbf860 con 0x558c6f8994d0 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f447ddcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c6fbc38a0 con 0x558c6f8ba1b0 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8912f0 0x558c6fc5dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52836/0 (socket says 172.21.15.50:52836) 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 -- 172.21.15.50:0/2405807239 learned_addr learned my addr 172.21.15.50:0/2405807239 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f446dffb640 1 -- 172.21.15.50:0/1974544495 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f445c0046c0 con 0x7f4470106600 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 -- 172.21.15.50:0/2405807239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 msgr2=0x558c6fc64c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc64c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 -- 172.21.15.50:0/2405807239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8912f0 msgr2=0x558c6fc5dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8912f0 0x558c6fc5dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 -- 172.21.15.50:0/2405807239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6fb87d50 con 0x558c6f8ba1b0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.355+0000 7f44753d9640 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4468009fd0 tx=0x7f44680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee085ab794d9e5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f445affd640 1 -- 172.21.15.50:0/2405807239 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468004d20 con 0x558c6f8ba1b0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f445affd640 1 -- 172.21.15.50:0/2405807239 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4468004050 con 0x558c6f8ba1b0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f445affd640 1 -- 172.21.15.50:0/2405807239 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468005260 con 0x558c6f8ba1b0 2024-05-17T04:39:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f446dffb640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 0x7f44600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f4474bd8640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 0x7f44600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.356+0000 7f446dffb640 1 -- 172.21.15.50:0/1974544495 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f445c0e28e0 con 0x7f4470106600 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- 172.21.15.50:0/2405807239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 msgr2=0x558c6fc5d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4468009fd0 tx=0x7f44680118d0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f4474bd8640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 0x7f44600b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4464005ab0 tx=0x7f4464005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- 172.21.15.50:0/2405807239 shutdown_connections 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc64c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8912f0 0x558c6fc5dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 --2- 172.21.15.50:0/2405807239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8ba1b0 0x558c6fc5d770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- 172.21.15.50:0/2405807239 >> 172.21.15.50:0/2405807239 conn(0x558c6f8ae300 msgr2=0x558c6fc591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- 172.21.15.50:0/2405807239 shutdown_connections 2024-05-17T04:39:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- 172.21.15.50:0/2405807239 wait complete. 2024-05-17T04:39:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 -- start start 2024-05-17T04:39:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.357+0000 7f447ddcc740 1 Processor -- start 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 0x558c6fc63ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c6fafbab0 con 0x558c6f8ba1b0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c6fb7c4d0 con 0x558c6f8994d0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558c6fc65510 con 0x558c6f8912f0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f447ddcc740 10 client.0 _ll_register_callbacks cb 0x558c6fbba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f446ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f44753d9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 0x558c6fc63ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f446ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59382/0 (socket says 172.21.15.50:59382) 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52848/0 (socket says 172.21.15.50:52848) 2024-05-17T04:39:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.358+0000 7f446ffff640 1 -- 172.21.15.50:0/4015636857 learned_addr learned my addr 172.21.15.50:0/4015636857 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 -- 172.21.15.50:0/4015636857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 msgr2=0x558c6fc62a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 -- 172.21.15.50:0/4015636857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 msgr2=0x558c6fc63ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 0x558c6fc63ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 -- 172.21.15.50:0/4015636857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6fb7d340 con 0x558c6f8994d0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f44753d9640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 0x558c6fc63ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f4474bd8640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f446ffff640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f445c004ab0 tx=0x7f445c0a92e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57151402f58e993 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f444ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0aa270 con 0x558c6f8994d0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4468004050 con 0x558c6f8994d0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f445c0b2750 con 0x558c6f8994d0 2024-05-17T04:39:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f445c0a9b70 con 0x558c6f8994d0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f447ddcc740 10 client.25307 fetch_fsmap learned FSMap version 33 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.359+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c6f925790 con 0x558c6f8994d0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4468004050 con 0x558c6f8994d0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f445c0a54e0 con 0x558c6f8994d0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 10 client.25307 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 20 client.25307 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f878d80 0x558c6fc7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 4 client.25307 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c6fbd1c10 con 0x558c6f878d80 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.360+0000 7f4474bd8640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f878d80 0x558c6fc7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.361+0000 7f4474bd8640 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f878d80 0x558c6fc7d0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd31f2353f7f205 server_cookie=43fac25d77f353db in_seq=0 out_seq=0 2024-05-17T04:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.361+0000 7f444ffff640 10 client.25307 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.361+0000 7f444ffff640 1 -- 172.21.15.50:0/4015636857 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c6fbd1c10 con 0x558c6f878d80 2024-05-17T04:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.361+0000 7f444ffff640 10 client.25307 handle_command_reply: tid=0 2024-05-17T04:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.361+0000 7f447ddcc740 1 client.25307 shutdown 2024-05-17T04:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 msgr2=0x558c6fc62fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f445c004ab0 tx=0x7f445c0a92e0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 shutdown_connections 2024-05-17T04:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f878d80 0x558c6fc7d0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f8ba1b0 0x558c6fc63ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f444ffff640 0 client.25307 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc62fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 --2- 172.21.15.50:0/4015636857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc62a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 >> 172.21.15.50:0/4015636857 conn(0x558c6f8ae300 msgr2=0x558c6fc581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 shutdown_connections 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 1 -- 172.21.15.50:0/4015636857 wait complete. 2024-05-17T04:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.362+0000 7f447ddcc740 20 client.25307 trim_cache size 0 max 16384 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 Processor -- start 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 -- start start 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44701ac690 con 0x558c6f878d80 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a9ea0 con 0x558c6f8994d0 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f447ddcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a7cf0 con 0x558c6f8912f0 2024-05-17T04:39:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f446ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f446ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52850/0 (socket says 172.21.15.50:52850) 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f446ffff640 1 -- 172.21.15.50:0/1057073322 learned_addr learned my addr 172.21.15.50:0/1057073322 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.380+0000 7f44753d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 -- 172.21.15.50:0/1057073322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 msgr2=0x558c6fc5c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc5c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 -- 172.21.15.50:0/1057073322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 msgr2=0x558c6fc59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 -- 172.21.15.50:0/1057073322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6f99e620 con 0x558c6f8912f0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f44753d9640 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f446ffff640 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f445c0a6de0 tx=0x7f445c0a9410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b019e530a7c6d358 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f445a7fc640 1 -- 172.21.15.50:0/1057073322 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0c8020 con 0x558c6f8912f0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f445a7fc640 1 -- 172.21.15.50:0/1057073322 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f445c0aa270 con 0x558c6f8912f0 2024-05-17T04:39:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.381+0000 7f445a7fc640 1 -- 172.21.15.50:0/1057073322 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a8840 con 0x558c6f8912f0 2024-05-17T04:39:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- 172.21.15.50:0/1057073322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 msgr2=0x558c6fc5bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f445c0a6de0 tx=0x7f445c0a9410 comp rx=0 tx=0).stop 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- 172.21.15.50:0/1057073322 shutdown_connections 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc5bb50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 --2- 172.21.15.50:0/1057073322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- 172.21.15.50:0/1057073322 >> 172.21.15.50:0/1057073322 conn(0x558c6f8ae300 msgr2=0x558c6fc58b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- 172.21.15.50:0/1057073322 shutdown_connections 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- 172.21.15.50:0/1057073322 wait complete. 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.382+0000 7f447ddcc740 1 -- start start 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 Processor -- start 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc75290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a8360 con 0x558c6f878d80 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a97d0 con 0x558c6f8994d0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a8a30 con 0x558c6f8912f0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f447ddcc740 10 client.0 _ll_register_callbacks cb 0x7f445c004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39924/0 (socket says 172.21.15.50:39924) 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 -- 172.21.15.50:0/3312221796 learned_addr learned my addr 172.21.15.50:0/3312221796 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f446ffff640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc75290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f44753d9640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 -- 172.21.15.50:0/3312221796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 msgr2=0x558c6fc75290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc75290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 -- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 msgr2=0x558c6fc761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.383+0000 7f4474bd8640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f4474bd8640 1 -- 172.21.15.50:0/3312221796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6fc767b0 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f446ffff640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc75290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f4474bd8640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f44640096c0 tx=0x7f4464009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8299bc5febd7a39c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f444ffff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464035040 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f44640538c0 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445c0aa270 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4464065050 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f447ddcc740 10 client.15555 fetch_fsmap learned FSMap version 33 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c6fc77ae0 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.384+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f445c0aa270 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f4464044040 con 0x558c6f878d80 2024-05-17T04:39:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 10 client.15555 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 20 client.15555 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f8ba1b0 0x558c6fce7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 4 client.15555 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c6fbcb3f0 con 0x558c6f8ba1b0 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f446ffff640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f8ba1b0 0x558c6fce7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f444ffff640 10 client.15555 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.385+0000 7f446ffff640 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f8ba1b0 0x558c6fce7950 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a533de58cbc845ba server_cookie=fc90b08f0c31d72 in_seq=0 out_seq=0 2024-05-17T04:39:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f444ffff640 1 -- 172.21.15.50:0/3312221796 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c6fbcb3f0 con 0x558c6f8ba1b0 2024-05-17T04:39:38.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f444ffff640 10 client.15555 handle_command_reply: tid=0 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 client.15555 shutdown 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 msgr2=0x558c6fc74d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f44640096c0 tx=0x7f4464009260 comp rx=0 tx=0).stop 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 shutdown_connections 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c6f8994d0 0x558c6fc761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c6f8912f0 0x558c6fc75290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558c6f8ba1b0 0x558c6fce7950 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 --2- 172.21.15.50:0/3312221796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c6f878d80 0x558c6fc74d50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f444ffff640 0 client.15555 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 >> 172.21.15.50:0/3312221796 conn(0x558c6f8ae300 msgr2=0x558c6fc577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 shutdown_connections 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 1 -- 172.21.15.50:0/3312221796 wait complete. 2024-05-17T04:39:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.453+0000 7f447ddcc740 20 client.15555 trim_cache size 0 max 16384 2024-05-17T04:39:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:39:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:39:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:39:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 msgr2=0x7f44600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 0x7f44600b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4464005ab0 tx=0x7f4464005fd0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 msgr2=0x7f4470199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f445c00e9f0 tx=0x7f445c00edf0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 shutdown_connections 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4470108f60 0x7f44701a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f44600b27e0 0x7f44600b4cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4470106600 0x7f4470199d40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 --2- 172.21.15.50:0/1974544495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4470103ca0 0x7f4470199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 >> 172.21.15.50:0/1974544495 conn(0x7f44700fd8b0 msgr2=0x7f4470104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 shutdown_connections 2024-05-17T04:39:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:38.454+0000 7f4476e3c640 1 -- 172.21.15.50:0/1974544495 wait complete. 2024-05-17T04:39:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:38 smithi105 ceph-mon[34795]: pgmap v538: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-05-17T04:39:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:38 smithi110 ceph-mon[35497]: pgmap v538: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-05-17T04:39:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:38 smithi050 ceph-mon[31739]: pgmap v538: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.5 MiB/s wr, 230 op/s 2024-05-17T04:39:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:40 smithi105 ceph-mon[34795]: pgmap v539: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:39:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:40 smithi110 ceph-mon[35497]: pgmap v539: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:39:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:40 smithi050 ceph-mon[31739]: pgmap v539: 129 pgs: 129 active+clean; 930 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:39:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:42 smithi105 ceph-mon[34795]: pgmap v540: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-05-17T04:39:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:42 smithi110 ceph-mon[35497]: pgmap v540: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-05-17T04:39:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:42 smithi050 ceph-mon[31739]: pgmap v540: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-05-17T04:39:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 45.0 s, 180.8 tps, lat 638.938 ms stddev 536.592 2024-05-17T04:39:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:44 smithi105 ceph-mon[34795]: pgmap v541: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.4 MiB/s wr, 383 op/s 2024-05-17T04:39:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:44 smithi110 ceph-mon[35497]: pgmap v541: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.4 MiB/s wr, 383 op/s 2024-05-17T04:39:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:44 smithi050 ceph-mon[31739]: pgmap v541: 129 pgs: 129 active+clean; 931 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.4 MiB/s wr, 383 op/s 2024-05-17T04:39:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:46 smithi105 ceph-mon[34795]: pgmap v542: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-05-17T04:39:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:46 smithi110 ceph-mon[35497]: pgmap v542: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-05-17T04:39:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:46 smithi050 ceph-mon[31739]: pgmap v542: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.8 MiB/s wr, 437 op/s 2024-05-17T04:39:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 50.0 s, 127.0 tps, lat 782.759 ms stddev 168.618 2024-05-17T04:39:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:48 smithi105 ceph-mon[34795]: pgmap v543: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-05-17T04:39:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:48 smithi110 ceph-mon[35497]: pgmap v543: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-05-17T04:39:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:48 smithi050 ceph-mon[31739]: pgmap v543: 129 pgs: 129 active+clean; 934 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-05-17T04:39:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:50 smithi105 ceph-mon[34795]: pgmap v544: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.7 MiB/s wr, 388 op/s 2024-05-17T04:39:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:50 smithi110 ceph-mon[35497]: pgmap v544: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.7 MiB/s wr, 388 op/s 2024-05-17T04:39:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:50 smithi050 ceph-mon[31739]: pgmap v544: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.7 MiB/s wr, 388 op/s 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 -- 172.21.15.50:0/279534656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4241031a0 msgr2=0x7fb4241035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 --2- 172.21.15.50:0/279534656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4241031a0 0x7fb4241035b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb410009900 tx=0x7fb41002a3f0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 -- 172.21.15.50:0/279534656 shutdown_connections 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 --2- 172.21.15.50:0/279534656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424103af0 0x7fb42410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 --2- 172.21.15.50:0/279534656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4241031a0 0x7fb4241035b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 --2- 172.21.15.50:0/279534656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb424102080 0x7fb424102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 -- 172.21.15.50:0/279534656 >> 172.21.15.50:0/279534656 conn(0x7fb4240fd880 msgr2=0x7fb4240ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 -- 172.21.15.50:0/279534656 shutdown_connections 2024-05-17T04:39:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.578+0000 7fb42b19a640 1 -- 172.21.15.50:0/279534656 wait complete. 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 Processor -- start 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 -- start start 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 0x7fb424198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb424103af0 0x7fb42419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb42410da60 con 0x7fb424102080 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb42410d8e0 con 0x7fb4241031a0 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb42b19a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb42410dbe0 con 0x7fb424103af0 2024-05-17T04:39:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb423fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb423fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45960/0 (socket says 172.21.15.50:45960) 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb423fff640 1 -- 172.21.15.50:0/500542510 learned_addr learned my addr 172.21.15.50:0/500542510 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb428f36640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 0x7fb424198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.579+0000 7fb429737640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb424103af0 0x7fb42419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 -- 172.21.15.50:0/500542510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb424103af0 msgr2=0x7fb42419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb424103af0 0x7fb42419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 -- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 msgr2=0x7fb424198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 0x7fb424198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 -- 172.21.15.50:0/500542510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4241a0260 con 0x7fb4241031a0 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb428f36640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 0x7fb424198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb423fff640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb410009230 tx=0x7fb410004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb421ffb640 1 -- 172.21.15.50:0/500542510 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41003f030 con 0x7fb4241031a0 2024-05-17T04:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4241a04f0 con 0x7fb4241031a0 2024-05-17T04:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb421ffb640 1 -- 172.21.15.50:0/500542510 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb410004440 con 0x7fb4241031a0 2024-05-17T04:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.580+0000 7fb421ffb640 1 -- 172.21.15.50:0/500542510 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410035070 con 0x7fb4241031a0 2024-05-17T04:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.581+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4241a09e0 con 0x7fb4241031a0 2024-05-17T04:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.583+0000 7fb421ffb640 1 -- 172.21.15.50:0/500542510 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb410042030 con 0x7fb4241031a0 2024-05-17T04:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.584+0000 7fb432052740 1 Processor -- start 2024-05-17T04:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.584+0000 7fb432052740 1 -- start start 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.584+0000 7fb421ffb640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 0x7fb4140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.584+0000 7fb432052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 0x55c9293f1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb432052740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb432052740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290304d0 0x55c9293f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb432052740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c929366cc0 con 0x55c9290282f0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb432052740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c929356860 con 0x55c9290304d0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb432052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c92935a8a0 con 0x55c9290511b0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb421ffb640 1 -- 172.21.15.50:0/500542510 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb410102a40 con 0x7fb4241031a0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb429737640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 0x7fb4140b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb428f36640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 0x55c9293f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb428f36640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 0x55c9293f1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:40332/0 (socket says 172.21.15.50:40332) 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb428f36640 1 -- 172.21.15.50:0/2794035021 learned_addr learned my addr 172.21.15.50:0/2794035021 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41894/0 (socket says 172.21.15.50:41894) 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb429737640 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290304d0 0x55c9293f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 -- 172.21.15.50:0/2794035021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 msgr2=0x55c9293f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 0x55c9293f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 -- 172.21.15.50:0/2794035021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290304d0 msgr2=0x55c9293f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290304d0 0x55c9293f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb423fff640 1 -- 172.21.15.50:0/2794035021 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92931ed50 con 0x55c9290282f0 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.585+0000 7fb429737640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 0x7fb4140b4cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005fd0 tx=0x7fb41c009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.586+0000 7fb423fff640 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb4101001b0 tx=0x7fb4100c5d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27338030af7c8b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.586+0000 7fb40effd640 1 -- 172.21.15.50:0/2794035021 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410049040 con 0x55c9290282f0 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.586+0000 7fb40effd640 1 -- 172.21.15.50:0/2794035021 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4100ce080 con 0x55c9290282f0 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.586+0000 7fb40effd640 1 -- 172.21.15.50:0/2794035021 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100d26a0 con 0x55c9290282f0 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 -- 172.21.15.50:0/2794035021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 msgr2=0x55c9293efb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb4101001b0 tx=0x7fb4100c5d70 comp rx=0 tx=0).stop 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 -- 172.21.15.50:0/2794035021 shutdown_connections 2024-05-17T04:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290304d0 0x55c9293f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c9293efb70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 --2- 172.21.15.50:0/2794035021 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290511b0 0x55c9293f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 -- 172.21.15.50:0/2794035021 >> 172.21.15.50:0/2794035021 conn(0x55c929045300 msgr2=0x55c9293f23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 -- 172.21.15.50:0/2794035021 shutdown_connections 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.587+0000 7fb432052740 1 -- 172.21.15.50:0/2794035021 wait complete. 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.588+0000 7fb432052740 1 -- start start 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.588+0000 7fb432052740 1 Processor -- start 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9294024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290511b0 0x55c929409420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c929292ab0 con 0x55c9290282f0 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9293134d0 con 0x55c9290511b0 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c929409990 con 0x55c9290304d0 2024-05-17T04:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb432052740 10 client.0 _ll_register_callbacks cb 0x55c929351950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41908/0 (socket says 172.21.15.50:41908) 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 -- 172.21.15.50:0/3625890673 learned_addr learned my addr 172.21.15.50:0/3625890673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb423fff640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9294024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb429737640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290511b0 0x55c929409420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 -- 172.21.15.50:0/3625890673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 msgr2=0x55c9294024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9294024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 -- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290511b0 msgr2=0x55c929409420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290511b0 0x55c929409420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 -- 172.21.15.50:0/3625890673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c929314340 con 0x55c9290282f0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb428f36640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb418012b10 tx=0x7fb41800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6738bf53e38450 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.589+0000 7fb403fff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800fb40 con 0x55c9290282f0 2024-05-17T04:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4100ce080 con 0x55c9290282f0 2024-05-17T04:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb41800fce0 con 0x55c9290282f0 2024-05-17T04:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb418025050 con 0x55c9290282f0 2024-05-17T04:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb432052740 10 client.15567 fetch_fsmap learned FSMap version 33 2024-05-17T04:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9290bc790 con 0x55c9290282f0 2024-05-17T04:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.590+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4180035f0 con 0x55c9290282f0 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb418010430 con 0x55c9290282f0 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 10 client.15567 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 20 client.15567 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c92900fd80 0x55c92940fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 4 client.15567 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb4100ce080 con 0x55c92900fd80 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb423fff640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c92900fd80 0x55c92940fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb403fff640 10 client.15567 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.591+0000 7fb423fff640 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c92900fd80 0x55c92940fdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c7f2dee5145b7eb server_cookie=46ae30fe116d7285 in_seq=0 out_seq=0 2024-05-17T04:39:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.592+0000 7fb403fff640 1 -- 172.21.15.50:0/3625890673 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb4100ce080 con 0x55c92900fd80 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.592+0000 7fb403fff640 10 client.15567 handle_command_reply: tid=0 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.592+0000 7fb432052740 1 client.15567 shutdown 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 msgr2=0x55c929401f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb418012b10 tx=0x7fb41800f3c0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 shutdown_connections 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290511b0 0x55c929409420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9294024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c92900fd80 0x55c92940fdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 --2- 172.21.15.50:0/3625890673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290282f0 0x55c929401f80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb403fff640 0 client.15567 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 >> 172.21.15.50:0/3625890673 conn(0x55c929045300 msgr2=0x55c9293f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 shutdown_connections 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 1 -- 172.21.15.50:0/3625890673 wait complete. 2024-05-17T04:39:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.593+0000 7fb432052740 20 client.15567 trim_cache size 0 max 16384 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.617+0000 7fb432052740 1 Processor -- start 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.617+0000 7fb432052740 1 -- start start 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9293fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9293fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241ac750 con 0x55c9290511b0 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241a9f60 con 0x55c9290282f0 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb432052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241a7db0 con 0x55c9290304d0 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb423fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9293fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41920/0 (socket says 172.21.15.50:41920) 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb428f36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9293fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 -- 172.21.15.50:0/1513804649 learned_addr learned my addr 172.21.15.50:0/1513804649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 -- 172.21.15.50:0/1513804649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 msgr2=0x55c9293fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9293fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 -- 172.21.15.50:0/1513804649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 msgr2=0x55c9293fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9293fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb429737640 1 -- 172.21.15.50:0/1513804649 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c929135620 con 0x55c9290511b0 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.618+0000 7fb423fff640 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9293fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.619+0000 7fb429737640 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb41c013380 tx=0x7fb41c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fd2de949afd555 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.619+0000 7fb40e7fc640 1 -- 172.21.15.50:0/1513804649 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004b20 con 0x55c9290511b0 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.619+0000 7fb40e7fc640 1 -- 172.21.15.50:0/1513804649 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb41c004cc0 con 0x55c9290511b0 2024-05-17T04:39:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.619+0000 7fb40e7fc640 1 -- 172.21.15.50:0/1513804649 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c005220 con 0x55c9290511b0 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 -- 172.21.15.50:0/1513804649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 msgr2=0x55c9294093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb41c013380 tx=0x7fb41c01f000 comp rx=0 tx=0).stop 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 -- 172.21.15.50:0/1513804649 shutdown_connections 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c9294093b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c9293fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 --2- 172.21.15.50:0/1513804649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9293fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 -- 172.21.15.50:0/1513804649 >> 172.21.15.50:0/1513804649 conn(0x55c929045300 msgr2=0x55c9293f1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 -- 172.21.15.50:0/1513804649 shutdown_connections 2024-05-17T04:39:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.620+0000 7fb432052740 1 -- 172.21.15.50:0/1513804649 wait complete. 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 -- start start 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 Processor -- start 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c929409be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c929406220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241a8420 con 0x55c9290511b0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241a9890 con 0x55c9290282f0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241a8af0 con 0x55c9290304d0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.621+0000 7fb432052740 10 client.0 _ll_register_callbacks cb 0x7fb418009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb429737640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c929406220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45984/0 (socket says 172.21.15.50:45984) 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 -- 172.21.15.50:0/2226454237 learned_addr learned my addr 172.21.15.50:0/2226454237 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:39:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb423fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c929409be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 -- 172.21.15.50:0/2226454237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 msgr2=0x55c929409be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c929409be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 -- 172.21.15.50:0/2226454237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 msgr2=0x55c929406220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c929406220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 -- 172.21.15.50:0/2226454237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c929406960 con 0x55c9290282f0 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb428f36640 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4180059f0 tx=0x7fb4180106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d57a838568611c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:39:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb403fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.622+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800fb40 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.623+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb418018040 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.623+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41c004cc0 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.623+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb418036050 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.623+0000 7fb432052740 10 client.25331 fetch_fsmap learned FSMap version 33 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.623+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c929476e10 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb41c004cc0 con 0x55c9290282f0 2024-05-17T04:39:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb418031030 con 0x55c9290282f0 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 10 client.25331 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 20 client.25331 populate_metadata read hostname 'smithi050' 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c928fe51c0 0x55c9294798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 4 client.25331 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.624+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c929368c10 con 0x55c928fe51c0 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.625+0000 7fb423fff640 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c928fe51c0 0x55c9294798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.625+0000 7fb403fff640 10 client.25331 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.625+0000 7fb423fff640 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c928fe51c0 0x55c9294798b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d6dbe26ea33ca2 server_cookie=2efb9a3ad4a7798c in_seq=0 out_seq=0 2024-05-17T04:39:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.641+0000 7fb403fff640 1 -- 172.21.15.50:0/2226454237 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c929368c10 con 0x55c928fe51c0 2024-05-17T04:39:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.641+0000 7fb403fff640 10 client.25331 handle_command_reply: tid=0 2024-05-17T04:39:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.641+0000 7fb432052740 1 client.25331 shutdown 2024-05-17T04:39:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 msgr2=0x55c9294096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4180059f0 tx=0x7fb4180106e0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 shutdown_connections 2024-05-17T04:39:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c928fe51c0 0x55c9294798b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9290511b0 0x55c929406220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9290304d0 0x55c929409be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 --2- 172.21.15.50:0/2226454237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9290282f0 0x55c9294096a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 >> 172.21.15.50:0/2226454237 conn(0x55c929045300 msgr2=0x55c9293fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb403fff640 0 client.25331 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 shutdown_connections 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 1 -- 172.21.15.50:0/2226454237 wait complete. 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.642+0000 7fb432052740 20 client.25331 trim_cache size 0 max 16384 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:39:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:39:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:39:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 msgr2=0x7fb4140b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 0x7fb4140b4cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005fd0 tx=0x7fb41c009b80 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 msgr2=0x7fb424198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb410009230 tx=0x7fb410004040 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 shutdown_connections 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb424103af0 0x7fb42419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4241031a0 0x7fb424198c60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4140b27e0 0x7fb4140b4cf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 --2- 172.21.15.50:0/500542510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb424102080 0x7fb424198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 >> 172.21.15.50:0/500542510 conn(0x7fb4240fd880 msgr2=0x7fb424106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 shutdown_connections 2024-05-17T04:39:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:39:51.643+0000 7fb42b19a640 1 -- 172.21.15.50:0/500542510 wait complete. 2024-05-17T04:39:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:52 smithi105 ceph-mon[34795]: pgmap v545: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 389 op/s 2024-05-17T04:39:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:52 smithi110 ceph-mon[35497]: pgmap v545: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 389 op/s 2024-05-17T04:39:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:52 smithi050 ceph-mon[31739]: pgmap v545: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 389 op/s 2024-05-17T04:39:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 55.0 s, 124.6 tps, lat 817.169 ms stddev 174.391 2024-05-17T04:39:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:54 smithi105 ceph-mon[34795]: pgmap v546: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:39:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:54 smithi110 ceph-mon[35497]: pgmap v546: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:39:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:54 smithi050 ceph-mon[31739]: pgmap v546: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:39:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:56 smithi105 ceph-mon[34795]: pgmap v547: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-17T04:39:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:56 smithi110 ceph-mon[35497]: pgmap v547: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-17T04:39:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:56 smithi050 ceph-mon[31739]: pgmap v547: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-05-17T04:39:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 60.0 s, 128.2 tps, lat 766.435 ms stddev 163.045 2024-05-17T04:39:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:39:58 smithi105 ceph-mon[34795]: pgmap v548: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-05-17T04:39:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:39:58 smithi110 ceph-mon[35497]: pgmap v548: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-05-17T04:39:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:39:58 smithi050 ceph-mon[31739]: pgmap v548: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.1 MiB/s wr, 312 op/s 2024-05-17T04:40:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:00 smithi105 ceph-mon[34795]: pgmap v549: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-05-17T04:40:00.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:00 smithi105 ceph-mon[34795]: overall HEALTH_OK 2024-05-17T04:40:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:00 smithi110 ceph-mon[35497]: pgmap v549: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-05-17T04:40:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:00 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:40:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:00 smithi050 ceph-mon[31739]: pgmap v549: 129 pgs: 129 active+clean; 935 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-05-17T04:40:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:00 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:40:02.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:02 smithi105 ceph-mon[34795]: pgmap v550: 129 pgs: 129 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-05-17T04:40:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:02 smithi110 ceph-mon[35497]: pgmap v550: 129 pgs: 129 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-05-17T04:40:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:02 smithi050 ceph-mon[31739]: pgmap v550: 129 pgs: 129 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.2 MiB/s wr, 366 op/s 2024-05-17T04:40:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 65.0 s, 129.4 tps, lat 779.844 ms stddev 156.610 2024-05-17T04:40:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:04 smithi105 ceph-mon[34795]: pgmap v551: 129 pgs: 129 active+clean; 935 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-17T04:40:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:04 smithi110 ceph-mon[35497]: pgmap v551: 129 pgs: 129 active+clean; 935 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-17T04:40:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:04 smithi050 ceph-mon[31739]: pgmap v551: 129 pgs: 129 active+clean; 935 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.6 MiB/s wr, 375 op/s 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.772+0000 7fd99e474640 1 -- 172.21.15.50:0/1870599154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998103580 msgr2=0x7fd998103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.772+0000 7fd99e474640 1 --2- 172.21.15.50:0/1870599154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998103580 0x7fd998103990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd988009860 tx=0x7fd98802a340 comp rx=0 tx=0).stop 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 -- 172.21.15.50:0/1870599154 shutdown_connections 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 --2- 172.21.15.50:0/1870599154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998102bd0 0x7fd99810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 --2- 172.21.15.50:0/1870599154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd998102280 0x7fd998102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 --2- 172.21.15.50:0/1870599154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998103580 0x7fd998103990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 -- 172.21.15.50:0/1870599154 >> 172.21.15.50:0/1870599154 conn(0x7fd9980fd030 msgr2=0x7fd9980ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 -- 172.21.15.50:0/1870599154 shutdown_connections 2024-05-17T04:40:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 -- 172.21.15.50:0/1870599154 wait complete. 2024-05-17T04:40:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.773+0000 7fd99e474640 1 Processor -- start 2024-05-17T04:40:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 -- start start 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd998102bd0 0x7fd998198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 0x7fd99819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd99810da70 con 0x7fd998103580 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd99810d8f0 con 0x7fd998102280 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99e474640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd99810dbf0 con 0x7fd998102bd0 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 0x7fd99819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37096/0 (socket says 172.21.15.50:37096) 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.774+0000 7fd997fff640 1 -- 172.21.15.50:0/1980612682 learned_addr learned my addr 172.21.15.50:0/1980612682 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 -- 172.21.15.50:0/1980612682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd998102bd0 msgr2=0x7fd998198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd998102bd0 0x7fd998198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 -- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 msgr2=0x7fd99819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 0x7fd99819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 -- 172.21.15.50:0/1980612682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9981a0250 con 0x7fd998102280 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd99ca11640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 0x7fd99819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd997fff640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd98802a930 tx=0x7fd98802ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd9957fa640 1 -- 172.21.15.50:0/1980612682 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988040030 con 0x7fd998102280 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd9957fa640 1 -- 172.21.15.50:0/1980612682 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9880305f0 con 0x7fd998102280 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd9957fa640 1 -- 172.21.15.50:0/1980612682 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988037040 con 0x7fd998102280 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9981a04e0 con 0x7fd998102280 2024-05-17T04:40:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.775+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9981a09d0 con 0x7fd998102280 2024-05-17T04:40:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.778+0000 7fd9957fa640 1 -- 172.21.15.50:0/1980612682 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd988043070 con 0x7fd998102280 2024-05-17T04:40:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.778+0000 7fd9a5408740 1 Processor -- start 2024-05-17T04:40:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.778+0000 7fd9a5408740 1 -- start start 2024-05-17T04:40:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb5f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb5ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb66e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5570bdacfcc0 con 0x5570bd7994d0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5570bdabf860 con 0x5570bd7912f0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9a5408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5570bdac38a0 con 0x5570bd7ba1b0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb5ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb5f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb5f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37414/0 (socket says 172.21.15.50:37414) 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd99ca11640 1 -- 172.21.15.50:0/1594917542 learned_addr learned my addr 172.21.15.50:0/1594917542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd9977fe640 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb66e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 -- 172.21.15.50:0/1594917542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 msgr2=0x5570bdb5f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb5f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 -- 172.21.15.50:0/1594917542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 msgr2=0x5570bdb66e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb66e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.779+0000 7fd997fff640 1 -- 172.21.15.50:0/1594917542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bda87d50 con 0x5570bd7912f0 2024-05-17T04:40:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd9957fa640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 0x7fd9800b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd997fff640 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb5ff00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9880ffb60 tx=0x7fd98810a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be28e782e64dc51d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd99ca11640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 0x7fd9800b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd97e7fc640 1 -- 172.21.15.50:0/1594917542 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880c5830 con 0x5570bd7912f0 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd97e7fc640 1 -- 172.21.15.50:0/1594917542 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd98810f030 con 0x5570bd7912f0 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.780+0000 7fd97e7fc640 1 -- 172.21.15.50:0/1594917542 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880c8f00 con 0x5570bd7912f0 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9957fa640 1 -- 172.21.15.50:0/1980612682 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd988102380 con 0x7fd998102280 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 -- 172.21.15.50:0/1594917542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 msgr2=0x5570bdb5ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd99ca11640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 0x7fd9800b4cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5570bdb60b70 tx=0x7fd990010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb5ff00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9880ffb60 tx=0x7fd98810a4d0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 -- 172.21.15.50:0/1594917542 shutdown_connections 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb66e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb5ff00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 --2- 172.21.15.50:0/1594917542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb5f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.781+0000 7fd9a5408740 1 -- 172.21.15.50:0/1594917542 >> 172.21.15.50:0/1594917542 conn(0x5570bd7ae300 msgr2=0x5570bdb591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.782+0000 7fd9a5408740 1 -- 172.21.15.50:0/1594917542 shutdown_connections 2024-05-17T04:40:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.782+0000 7fd9a5408740 1 -- 172.21.15.50:0/1594917542 wait complete. 2024-05-17T04:40:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.782+0000 7fd9a5408740 1 -- start start 2024-05-17T04:40:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.782+0000 7fd9a5408740 1 Processor -- start 2024-05-17T04:40:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb71e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56390/0 (socket says 172.21.15.50:56390) 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 -- 172.21.15.50:0/2999975690 learned_addr learned my addr 172.21.15.50:0/2999975690 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb76920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5570bd9fbab0 con 0x5570bd7994d0 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5570bda7c4d0 con 0x5570bd7912f0 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5570bdabf860 con 0x5570bd7ba1b0 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd997fff640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb71e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd99ca11640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb76920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 -- 172.21.15.50:0/2999975690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 msgr2=0x5570bdb76920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb76920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 -- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 msgr2=0x5570bdb71e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.783+0000 7fd9977fe640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb71e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9977fe640 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bda7d340 con 0x5570bd7994d0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9a5408740 10 client.0 _ll_register_callbacks cb 0x5570bdaba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98810f030 con 0x5570bd7994d0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd997fff640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb71e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9977fe640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009aa0 tx=0x7fd98c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d771cf4f88e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9777fe640 10 client.15585 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9777fe640 1 -- 172.21.15.50:0/2999975690 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01f060 con 0x5570bd7994d0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9777fe640 1 -- 172.21.15.50:0/2999975690 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd98c00eb40 con 0x5570bd7994d0 2024-05-17T04:40:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9777fe640 1 -- 172.21.15.50:0/2999975690 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd98c00ed40 con 0x5570bd7994d0 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9a5408740 10 client.15585 fetch_fsmap learned FSMap version 33 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.784+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=33}) -- 0x5570bd825790 con 0x5570bd7994d0 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9777fe640 1 -- 172.21.15.50:0/2999975690 <== mon.0 v2:172.21.15.50:3300/0 4 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fd98c01a440 con 0x5570bd7994d0 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 10 client.15585 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 20 client.15585 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd778d80 0x5570bdb7c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 4 client.15585 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570bdad1c10 con 0x5570bd778d80 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.785+0000 7fd997fff640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd778d80 0x5570bdb7c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.786+0000 7fd997fff640 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd778d80 0x5570bdb7c9f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6b825504c79896 server_cookie=c34a2bab67cc793c in_seq=0 out_seq=0 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.786+0000 7fd9777fe640 10 client.15585 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.786+0000 7fd9777fe640 1 -- 172.21.15.50:0/2999975690 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570bdad1c10 con 0x5570bd778d80 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.786+0000 7fd9777fe640 10 client.15585 handle_command_reply: tid=0 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.786+0000 7fd9a5408740 1 client.15585 shutdown 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 msgr2=0x5570bdb743d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009aa0 tx=0x7fd98c009f60 comp rx=0 tx=0).stop 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 shutdown_connections 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7ba1b0 0x5570bdb76920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd778d80 0x5570bdb7c9f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb743d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2999975690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd7912f0 0x5570bdb71e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9777fe640 0 client.15585 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 >> 172.21.15.50:0/2999975690 conn(0x5570bd7ae300 msgr2=0x5570bdb577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 shutdown_connections 2024-05-17T04:40:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 1 -- 172.21.15.50:0/2999975690 wait complete. 2024-05-17T04:40:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.787+0000 7fd9a5408740 20 client.15585 trim_cache size 0 max 16384 2024-05-17T04:40:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 Processor -- start 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 -- start start 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981ac6a0 con 0x5570bd7994d0 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.809+0000 7fd9a5408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981a9eb0 con 0x5570bd778d80 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9a5408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981a7d00 con 0x5570bd7912f0 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37440/0 (socket says 172.21.15.50:37440) 2024-05-17T04:40:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37132/0 (socket says 172.21.15.50:37132) 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56398/0 (socket says 172.21.15.50:56398) 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd997fff640 1 -- 172.21.15.50:0/3153284998 learned_addr learned my addr 172.21.15.50:0/3153284998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 -- 172.21.15.50:0/3153284998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 msgr2=0x5570bdb65e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 -- 172.21.15.50:0/3153284998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 msgr2=0x5570bdb614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 -- 172.21.15.50:0/3153284998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bd89e620 con 0x5570bd7912f0 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd997fff640 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd99ca11640 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.810+0000 7fd9977fe640 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009aa0 tx=0x7fd98c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534fb2945d6cf87b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd97dffb640 1 -- 172.21.15.50:0/3153284998 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c01f040 con 0x5570bd7912f0 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd97dffb640 1 -- 172.21.15.50:0/3153284998 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd98c010e50 con 0x5570bd7912f0 2024-05-17T04:40:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd97dffb640 1 -- 172.21.15.50:0/3153284998 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00e940 con 0x5570bd7912f0 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 -- 172.21.15.50:0/3153284998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 msgr2=0x5570bdb5c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009aa0 tx=0x7fd98c014000 comp rx=0 tx=0).stop 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 -- 172.21.15.50:0/3153284998 shutdown_connections 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb5c860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.811+0000 7fd9a5408740 1 --2- 172.21.15.50:0/3153284998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb65e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 -- 172.21.15.50:0/3153284998 >> 172.21.15.50:0/3153284998 conn(0x5570bd7ae300 msgr2=0x5570bdb60db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 -- 172.21.15.50:0/3153284998 shutdown_connections 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 -- 172.21.15.50:0/3153284998 wait complete. 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 -- start start 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 Processor -- start 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.812+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb74ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981a8370 con 0x5570bd7994d0 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981a97e0 con 0x5570bd778d80 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981a8a40 con 0x5570bd7912f0 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9a5408740 10 client.0 _ll_register_callbacks cb 0x7fd98c0108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd99ca11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb75910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56404/0 (socket says 172.21.15.50:56404) 2024-05-17T04:40:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37136/0 (socket says 172.21.15.50:37136) 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd99ca11640 1 -- 172.21.15.50:0/2857230192 learned_addr learned my addr 172.21.15.50:0/2857230192 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9977fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb74ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 -- 172.21.15.50:0/2857230192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 msgr2=0x5570bdb74ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb74ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 -- 172.21.15.50:0/2857230192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 msgr2=0x5570bdb75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb75910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 -- 172.21.15.50:0/2857230192 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bdb75f00 con 0x5570bd778d80 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd997fff640 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c9670 tx=0x7fd988005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ddb682587b90445 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.813+0000 7fd9777fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880c90e0 con 0x5570bd778d80 2024-05-17T04:40:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd9880ffca0 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c010e50 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd9880c92c0 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 10 client.25349 fetch_fsmap learned FSMap version 33 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570bdb77190 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd98c010e50 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fd98804d060 con 0x5570bd778d80 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 10 client.25349 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 10 client.25349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.814+0000 7fd9a5408740 20 client.25349 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.815+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd7ba1b0 0x5570bdbe7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.815+0000 7fd9a5408740 4 client.25349 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.815+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570bdacb3f0 con 0x5570bd7ba1b0 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.815+0000 7fd9977fe640 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd7ba1b0 0x5570bdbe7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.815+0000 7fd9777fe640 10 client.25349 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.816+0000 7fd9977fe640 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd7ba1b0 0x5570bdbe7140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e899ae57e0afa49 server_cookie=2d9924b50c0ded64 in_seq=0 out_seq=0 2024-05-17T04:40:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.828+0000 7fd9777fe640 1 -- 172.21.15.50:0/2857230192 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5570bdacb3f0 con 0x5570bd7ba1b0 2024-05-17T04:40:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.828+0000 7fd9777fe640 10 client.25349 handle_command_reply: tid=0 2024-05-17T04:40:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 client.25349 shutdown 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 msgr2=0x5570bdb74580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c9670 tx=0x7fd988005bd0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 shutdown_connections 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5570bd7ba1b0 0x5570bdbe7140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5570bd7994d0 0x5570bdb75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5570bd7912f0 0x5570bdb74ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 --2- 172.21.15.50:0/2857230192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570bd778d80 0x5570bdb74580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 >> 172.21.15.50:0/2857230192 conn(0x5570bd7ae300 msgr2=0x5570bdb5c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.829+0000 7fd9777fe640 0 client.25349 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.830+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 shutdown_connections 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.830+0000 7fd9a5408740 1 -- 172.21.15.50:0/2857230192 wait complete. 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.830+0000 7fd9a5408740 20 client.25349 trim_cache size 0 max 16384 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:40:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 msgr2=0x7fd9800b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 0x7fd9800b4cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5570bdb60b70 tx=0x7fd990010040 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 msgr2=0x7fd998198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd98802a930 tx=0x7fd98802ad70 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 shutdown_connections 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd9800b27e0 0x7fd9800b4cf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd998103580 0x7fd99819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd998102bd0 0x7fd998198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 --2- 172.21.15.50:0/1980612682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd998102280 0x7fd998198710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 >> 172.21.15.50:0/1980612682 conn(0x7fd9980fd030 msgr2=0x7fd998106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 shutdown_connections 2024-05-17T04:40:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:04.831+0000 7fd99e474640 1 -- 172.21.15.50:0/1980612682 wait complete. 2024-05-17T04:40:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:06 smithi105 ceph-mon[34795]: pgmap v552: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-05-17T04:40:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:06 smithi110 ceph-mon[35497]: pgmap v552: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-05-17T04:40:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:06 smithi050 ceph-mon[31739]: pgmap v552: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-05-17T04:40:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 70.0 s, 124.0 tps, lat 796.965 ms stddev 159.746 2024-05-17T04:40:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:08 smithi105 ceph-mon[34795]: pgmap v553: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:40:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:08 smithi110 ceph-mon[35497]: pgmap v553: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:40:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:08 smithi050 ceph-mon[31739]: pgmap v553: 129 pgs: 129 active+clean; 936 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:40:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:10 smithi105 ceph-mon[34795]: pgmap v554: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-05-17T04:40:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:10 smithi110 ceph-mon[35497]: pgmap v554: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-05-17T04:40:10.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:10 smithi050 ceph-mon[31739]: pgmap v554: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 411 op/s 2024-05-17T04:40:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:12 smithi105 ceph-mon[34795]: pgmap v555: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-05-17T04:40:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:12 smithi110 ceph-mon[35497]: pgmap v555: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-05-17T04:40:12.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:12 smithi050 ceph-mon[31739]: pgmap v555: 129 pgs: 129 active+clean; 936 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 343 op/s 2024-05-17T04:40:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 75.0 s, 123.8 tps, lat 811.111 ms stddev 167.077 2024-05-17T04:40:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:14 smithi105 ceph-mon[34795]: pgmap v556: 129 pgs: 129 active+clean; 937 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 418 op/s 2024-05-17T04:40:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:14 smithi110 ceph-mon[35497]: pgmap v556: 129 pgs: 129 active+clean; 937 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 418 op/s 2024-05-17T04:40:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:14 smithi050 ceph-mon[31739]: pgmap v556: 129 pgs: 129 active+clean; 937 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 418 op/s 2024-05-17T04:40:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:16 smithi105 ceph-mon[34795]: pgmap v557: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-05-17T04:40:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:16 smithi110 ceph-mon[35497]: pgmap v557: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-05-17T04:40:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:16 smithi050 ceph-mon[31739]: pgmap v557: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.2 MiB/s wr, 373 op/s 2024-05-17T04:40:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 80.0 s, 121.8 tps, lat 817.783 ms stddev 161.716 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 -- 172.21.15.50:0/1950449174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2681062b0 msgr2=0x7fb268108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/1950449174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2681062b0 0x7fb268108790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009900 tx=0x7fb25c02a410 comp rx=0 tx=0).stop 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 -- 172.21.15.50:0/1950449174 shutdown_connections 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/1950449174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb268108e20 0x7fb26810b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/1950449174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2681062b0 0x7fb268108790 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/1950449174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb26806b230 0x7fb268105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 -- 172.21.15.50:0/1950449174 >> 172.21.15.50:0/1950449174 conn(0x7fb2680fd880 msgr2=0x7fb2680ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 -- 172.21.15.50:0/1950449174 shutdown_connections 2024-05-17T04:40:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.960+0000 7fb26e3aa640 1 -- 172.21.15.50:0/1950449174 wait complete. 2024-05-17T04:40:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.961+0000 7fb26e3aa640 1 Processor -- start 2024-05-17T04:40:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.961+0000 7fb26e3aa640 1 -- start start 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 0x7fb268103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 0x7fb268101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 0x7fb268102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810eb50 con 0x7fb268108e20 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810e9d0 con 0x7fb26806b230 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26e3aa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810ecd0 con 0x7fb2681062b0 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb267fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 0x7fb268103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 0x7fb268101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb267fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 0x7fb268103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53526/0 (socket says 172.21.15.50:53526) 2024-05-17T04:40:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb267fff640 1 -- 172.21.15.50:0/2649238662 learned_addr learned my addr 172.21.15.50:0/2649238662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 0x7fb268102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 -- 172.21.15.50:0/2649238662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 msgr2=0x7fb268101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 0x7fb268101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 -- 172.21.15.50:0/2649238662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 msgr2=0x7fb268103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 0x7fb268103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb26c947640 1 -- 172.21.15.50:0/2649238662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268102850 con 0x7fb268108e20 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.962+0000 7fb2677fe640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 0x7fb268101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 0x7fb268102120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb26000da10 tx=0x7fb26000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb2657fa640 1 -- 172.21.15.50:0/2649238662 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26000bd70 con 0x7fb268108e20 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb2657fa640 1 -- 172.21.15.50:0/2649238662 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb260003e00 con 0x7fb268108e20 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb2657fa640 1 -- 172.21.15.50:0/2649238662 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600106c0 con 0x7fb268108e20 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681a3910 con 0x7fb268108e20 2024-05-17T04:40:17.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.963+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681a3d80 con 0x7fb268108e20 2024-05-17T04:40:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb2657fa640 1 -- 172.21.15.50:0/2649238662 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb260003970 con 0x7fb268108e20 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 Processor -- start 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 -- start start 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591cd646cc0 con 0x5591cd3311b0 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591cd636860 con 0x5591cd3082f0 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.967+0000 7fb275262740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591cd63a8a0 con 0x5591cd3104d0 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53388/0 (socket says 172.21.15.50:53388) 2024-05-17T04:40:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2677fe640 1 -- 172.21.15.50:0/93088709 learned_addr learned my addr 172.21.15.50:0/93088709 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2677fe640 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2657fa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 0x7fb2500b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 0x7fb2500b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 -- 172.21.15.50:0/93088709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 msgr2=0x5591cd6dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 -- 172.21.15.50:0/93088709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 msgr2=0x5591cd6d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb267fff640 1 -- 172.21.15.50:0/93088709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591cd5fed50 con 0x5591cd3082f0 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2677fe640 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.968+0000 7fb2657fa640 1 -- 172.21.15.50:0/2649238662 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb2600e2840 con 0x7fb268108e20 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.969+0000 7fb267fff640 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d4cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb258009c80 tx=0x7fb258011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9344af72ba79e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.969+0000 7fb26c947640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 0x7fb2500b4e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb2600a5b50 tx=0x7fb2600d90e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.969+0000 7fb24e7fc640 1 -- 172.21.15.50:0/93088709 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25801e070 con 0x5591cd3082f0 2024-05-17T04:40:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.969+0000 7fb24e7fc640 1 -- 172.21.15.50:0/93088709 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb258009e30 con 0x5591cd3082f0 2024-05-17T04:40:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.969+0000 7fb24e7fc640 1 -- 172.21.15.50:0/93088709 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25800e900 con 0x5591cd3082f0 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 -- 172.21.15.50:0/93088709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 msgr2=0x5591cd6d4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d4cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb258009c80 tx=0x7fb258011810 comp rx=0 tx=0).stop 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 -- 172.21.15.50:0/93088709 shutdown_connections 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d4cf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 --2- 172.21.15.50:0/93088709 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 -- 172.21.15.50:0/93088709 >> 172.21.15.50:0/93088709 conn(0x5591cd325300 msgr2=0x5591cd6d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 -- 172.21.15.50:0/93088709 shutdown_connections 2024-05-17T04:40:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.970+0000 7fb275262740 1 -- 172.21.15.50:0/93088709 wait complete. 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.971+0000 7fb275262740 1 -- start start 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.971+0000 7fb275262740 1 Processor -- start 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3082f0 0x5591cd6de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591cd572ab0 con 0x5591cd3104d0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591cd5f34d0 con 0x5591cd3311b0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591cd6dc690 con 0x5591cd3082f0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb275262740 10 client.0 _ll_register_callbacks cb 0x5591cd631950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb267fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb267fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53400/0 (socket says 172.21.15.50:53400) 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53536/0 (socket says 172.21.15.50:53536) 2024-05-17T04:40:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb267fff640 1 -- 172.21.15.50:0/1123057 learned_addr learned my addr 172.21.15.50:0/1123057 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3082f0 0x5591cd6de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 -- 172.21.15.50:0/1123057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3082f0 msgr2=0x5591cd6de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3082f0 0x5591cd6de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 -- 172.21.15.50:0/1123057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 msgr2=0x5591cd6de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb2677fe640 1 -- 172.21.15.50:0/1123057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591cd5f4340 con 0x5591cd3311b0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.972+0000 7fb267fff640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2677fe640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0319f0 tx=0x7fb25c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383d6e24b11f82af server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c035070 con 0x5591cd3311b0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb25c004260 con 0x5591cd3311b0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb258009e30 con 0x5591cd3311b0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb25c004460 con 0x5591cd3311b0 2024-05-17T04:40:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb275262740 10 client.25361 fetch_fsmap learned FSMap version 33 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591cd39c790 con 0x5591cd3311b0 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb258009e30 con 0x5591cd3311b0 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.973+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb25c003940 con 0x5591cd3311b0 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 10 client.25361 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 20 client.25361 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2efd80 0x5591cd6eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 4 client.25361 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591cd648c10 con 0x5591cd2efd80 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb267fff640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2efd80 0x5591cd6eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb2477fe640 10 client.25361 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.974+0000 7fb267fff640 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2efd80 0x5591cd6eb7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337df2e5d5b4a09a server_cookie=d823cfe54dbff709 in_seq=0 out_seq=0 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb2477fe640 1 -- 172.21.15.50:0/1123057 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591cd648c10 con 0x5591cd2efd80 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb2477fe640 10 client.25361 handle_command_reply: tid=0 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 client.25361 shutdown 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 msgr2=0x5591cd6e5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0319f0 tx=0x7fb25c000bc0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 shutdown_connections 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e5500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2efd80 0x5591cd6eb7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3104d0 0x5591cd6de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 --2- 172.21.15.50:0/1123057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3082f0 0x5591cd6de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb2477fe640 0 client.25361 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.975+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 >> 172.21.15.50:0/1123057 conn(0x5591cd325300 msgr2=0x5591cd6d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.976+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 shutdown_connections 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.976+0000 7fb275262740 1 -- 172.21.15.50:0/1123057 wait complete. 2024-05-17T04:40:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:17.976+0000 7fb275262740 20 client.25361 trim_cache size 0 max 16384 2024-05-17T04:40:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.000+0000 7fb275262740 1 Processor -- start 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 -- start start 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b0b00 con 0x5591cd3311b0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681ae310 con 0x5591cd3082f0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb275262740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681ac160 con 0x5591cd3104d0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53412/0 (socket says 172.21.15.50:53412) 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb2677fe640 1 -- 172.21.15.50:0/623361180 learned_addr learned my addr 172.21.15.50:0/623361180 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb2677fe640 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 -- 172.21.15.50:0/623361180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 msgr2=0x5591cd6d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 -- 172.21.15.50:0/623361180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 msgr2=0x5591cd6d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.001+0000 7fb267fff640 1 -- 172.21.15.50:0/623361180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591cd415620 con 0x5591cd3104d0 2024-05-17T04:40:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.002+0000 7fb267fff640 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6d2b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5591cd6ebf90 tx=0x7fb258005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbeec97f84254b51 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.002+0000 7fb24dffb640 1 -- 172.21.15.50:0/623361180 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25801e040 con 0x5591cd3104d0 2024-05-17T04:40:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.002+0000 7fb24dffb640 1 -- 172.21.15.50:0/623361180 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb25801f040 con 0x5591cd3104d0 2024-05-17T04:40:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.002+0000 7fb24dffb640 1 -- 172.21.15.50:0/623361180 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258004e90 con 0x5591cd3104d0 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 -- 172.21.15.50:0/623361180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 msgr2=0x5591cd6d2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6d2b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5591cd6ebf90 tx=0x7fb258005d30 comp rx=0 tx=0).stop 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 -- 172.21.15.50:0/623361180 shutdown_connections 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3311b0 0x5591cd6d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6d2b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 --2- 172.21.15.50:0/623361180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3082f0 0x5591cd6d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 -- 172.21.15.50:0/623361180 >> 172.21.15.50:0/623361180 conn(0x5591cd325300 msgr2=0x5591cd6cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 -- 172.21.15.50:0/623361180 shutdown_connections 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.003+0000 7fb275262740 1 -- 172.21.15.50:0/623361180 wait complete. 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.004+0000 7fb275262740 1 -- start start 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.004+0000 7fb275262740 1 Processor -- start 2024-05-17T04:40:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.004+0000 7fb275262740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53428/0 (socket says 172.21.15.50:53428) 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 -- 172.21.15.50:0/4259930717 learned_addr learned my addr 172.21.15.50:0/4259930717 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681ac7d0 con 0x5591cd3082f0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681adc40 con 0x5591cd3311b0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681acea0 con 0x5591cd3104d0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb275262740 10 client.0 _ll_register_callbacks cb 0x7fb25c032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb267fff640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 -- 172.21.15.50:0/4259930717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 msgr2=0x5591cd6e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 -- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 msgr2=0x5591cd6e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591cd751b90 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2677fe640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb25c02a930 tx=0x7fb25c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d4bd67ac6e63e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.005+0000 7fb2477fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c062060 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb25c03fd60 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25801f040 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb25c035040 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 10 client.15609 fetch_fsmap learned FSMap version 33 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591cd752d30 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb25801f040 con 0x5591cd3082f0 2024-05-17T04:40:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb25c03f0f0 con 0x5591cd3082f0 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 10 client.15609 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.006+0000 7fb275262740 20 client.15609 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2c51c0 0x5591cd755780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb275262740 4 client.15609 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591cd6423f0 con 0x5591cd2c51c0 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb267fff640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2c51c0 0x5591cd755780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb2477fe640 10 client.15609 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.007+0000 7fb267fff640 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2c51c0 0x5591cd755780 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e62b6396bc8cc00 server_cookie=f9ff4ec1e91ddca6 in_seq=0 out_seq=0 2024-05-17T04:40:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb2477fe640 1 -- 172.21.15.50:0/4259930717 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5591cd6423f0 con 0x5591cd2c51c0 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb2477fe640 10 client.15609 handle_command_reply: tid=0 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 client.15609 shutdown 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 msgr2=0x5591cd6e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb25c02a930 tx=0x7fb25c005d50 comp rx=0 tx=0).stop 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 shutdown_connections 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591cd3311b0 0x5591cd6e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591cd3104d0 0x5591cd6e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5591cd2c51c0 0x5591cd755780 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 --2- 172.21.15.50:0/4259930717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591cd3082f0 0x5591cd6e4a30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb2477fe640 0 client.15609 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:18.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 >> 172.21.15.50:0/4259930717 conn(0x5591cd325300 msgr2=0x5591cd6d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 shutdown_connections 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.030+0000 7fb275262740 1 -- 172.21.15.50:0/4259930717 wait complete. 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb275262740 20 client.15609 trim_cache size 0 max 16384 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:40:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 msgr2=0x7fb2500b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 0x7fb2500b4e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb2600a5b50 tx=0x7fb2600d90e0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 msgr2=0x7fb268102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 0x7fb268102120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb26000da10 tx=0x7fb26000ded0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 shutdown_connections 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb2500b2930 0x7fb2500b4e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb268108e20 0x7fb268102120 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2681062b0 0x7fb268101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 --2- 172.21.15.50:0/2649238662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb26806b230 0x7fb268103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 >> 172.21.15.50:0/2649238662 conn(0x7fb2680fd880 msgr2=0x7fb2681044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 shutdown_connections 2024-05-17T04:40:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:18.031+0000 7fb26e3aa640 1 -- 172.21.15.50:0/2649238662 wait complete. 2024-05-17T04:40:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:18 smithi105 ceph-mon[34795]: pgmap v558: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-05-17T04:40:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:18 smithi110 ceph-mon[35497]: pgmap v558: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-05-17T04:40:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:18 smithi050 ceph-mon[31739]: pgmap v558: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-05-17T04:40:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:20 smithi105 ceph-mon[34795]: pgmap v559: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.5 MiB/s wr, 396 op/s 2024-05-17T04:40:20.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:20 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:20 smithi110 ceph-mon[35497]: pgmap v559: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.5 MiB/s wr, 396 op/s 2024-05-17T04:40:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:20 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:20 smithi050 ceph-mon[31739]: pgmap v559: 129 pgs: 129 active+clean; 937 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.5 MiB/s wr, 396 op/s 2024-05-17T04:40:20.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:20 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:21.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:21 smithi105 ceph-mon[34795]: pgmap v560: 129 pgs: 129 active+clean; 937 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:40:21.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:21 smithi110 ceph-mon[35497]: pgmap v560: 129 pgs: 129 active+clean; 937 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:40:21.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:21 smithi050 ceph-mon[31739]: pgmap v560: 129 pgs: 129 active+clean; 937 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:40:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 85.0 s, 111.2 tps, lat 891.732 ms stddev 177.383 2024-05-17T04:40:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:23 smithi105 ceph-mon[34795]: pgmap v561: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-05-17T04:40:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:23 smithi110 ceph-mon[35497]: pgmap v561: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-05-17T04:40:24.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:23 smithi050 ceph-mon[31739]: pgmap v561: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.5 MiB/s wr, 399 op/s 2024-05-17T04:40:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:25 smithi105 ceph-mon[34795]: pgmap v562: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-05-17T04:40:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:25 smithi110 ceph-mon[35497]: pgmap v562: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-05-17T04:40:26.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:25 smithi050 ceph-mon[31739]: pgmap v562: 129 pgs: 129 active+clean; 938 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-05-17T04:40:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 90.0 s, 115.2 tps, lat 870.216 ms stddev 195.669 2024-05-17T04:40:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:27 smithi105 ceph-mon[34795]: pgmap v563: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-05-17T04:40:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:27 smithi110 ceph-mon[35497]: pgmap v563: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-05-17T04:40:28.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:27 smithi050 ceph-mon[31739]: pgmap v563: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-05-17T04:40:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:29 smithi105 ceph-mon[34795]: pgmap v564: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:40:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:29 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:40:30.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:29 smithi110 ceph-mon[35497]: pgmap v564: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:40:30.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:29 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:40:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:29 smithi050 ceph-mon[31739]: pgmap v564: 129 pgs: 129 active+clean; 938 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:40:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:29 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.171+0000 7f03a4a8c640 1 -- 172.21.15.50:0/2417898204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a0108f30 msgr2=0x7f03a010b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.171+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/2417898204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a0108f30 0x7f03a010b350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0398009be0 tx=0x7f039802a6c0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 -- 172.21.15.50:0/2417898204 shutdown_connections 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/2417898204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a0108f30 0x7f03a010b350 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/2417898204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a01065d0 0x7f03a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/2417898204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0103c70 0x7f03a0106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 -- 172.21.15.50:0/2417898204 >> 172.21.15.50:0/2417898204 conn(0x7f03a00fd880 msgr2=0x7f03a00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 -- 172.21.15.50:0/2417898204 shutdown_connections 2024-05-17T04:40:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.172+0000 7f03a4a8c640 1 -- 172.21.15.50:0/2417898204 wait complete. 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 Processor -- start 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 -- start start 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a01065d0 0x7f03a0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0108f30 0x7f03a01a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010eb50 con 0x7f03a01065d0 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010e9d0 con 0x7f03a0103c70 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.173+0000 7f03a4a8c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010ecd0 con 0x7f03a0108f30 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:43468/0 (socket says 172.21.15.50:43468) 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 -- 172.21.15.50:0/4060083813 learned_addr learned my addr 172.21.15.50:0/4060083813 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039dd9b640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a01065d0 0x7f03a0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039ed9d640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0108f30 0x7f03a01a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 -- 172.21.15.50:0/4060083813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0108f30 msgr2=0x7f03a01a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0108f30 0x7f03a01a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 -- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a01065d0 msgr2=0x7f03a0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a01065d0 0x7f03a0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 -- 172.21.15.50:0/4060083813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a01a78c0 con 0x7f03a0103c70 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.174+0000 7f039e59c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f039000e9c0 tx=0x7f039000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.175+0000 7f038f7fe640 1 -- 172.21.15.50:0/4060083813 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039000ccc0 con 0x7f03a0103c70 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.175+0000 7f038f7fe640 1 -- 172.21.15.50:0/4060083813 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03900044b0 con 0x7f03a0103c70 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.175+0000 7f038f7fe640 1 -- 172.21.15.50:0/4060083813 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390003e50 con 0x7f03a0103c70 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.175+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a01a7bb0 con 0x7f03a0103c70 2024-05-17T04:40:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.175+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01a80b0 con 0x7f03a0103c70 2024-05-17T04:40:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.178+0000 7f03aba16740 1 Processor -- start 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.178+0000 7f03aba16740 1 -- start start 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f038f7fe640 1 -- 172.21.15.50:0/4060083813 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0390018430 con 0x7f03a0103c70 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4e41b0 0x55b88b887770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 0x55b88b887cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b88b7f9cc0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b88b7e9860 con 0x55b88b4c34d0 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f03aba16740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b88b7ed8a0 con 0x55b88b4e41b0 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 0x55b88b887cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:43476/0 (socket says 172.21.15.50:43476) 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4e41b0 0x55b88b887770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 -- 172.21.15.50:0/3177091060 learned_addr learned my addr 172.21.15.50:0/3177091060 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039e59c640 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 0x55b88b887cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38332/0 (socket says 172.21.15.50:38332) 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 -- 172.21.15.50:0/3177091060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4e41b0 msgr2=0x55b88b887770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4e41b0 0x55b88b887770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 -- 172.21.15.50:0/3177091060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 msgr2=0x55b88b887cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 0x55b88b887cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.179+0000 7f039dd9b640 1 -- 172.21.15.50:0/3177091060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88b7b1d50 con 0x55b88b4c34d0 2024-05-17T04:40:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f039dd9b640 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0394009cb0 tx=0x7f0394011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde4b0448a5164fc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f038f7fe640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 0x7f03880b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f038cff9640 1 -- 172.21.15.50:0/3177091060 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394004230 con 0x55b88b4c34d0 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f038cff9640 1 -- 172.21.15.50:0/3177091060 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03940043d0 con 0x55b88b4c34d0 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f038cff9640 1 -- 172.21.15.50:0/3177091060 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940196c0 con 0x55b88b4c34d0 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f039ed9d640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 0x7f03880b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.180+0000 7f038f7fe640 1 -- 172.21.15.50:0/4060083813 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f03900e29e0 con 0x7f03a0103c70 2024-05-17T04:40:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f03aba16740 1 -- 172.21.15.50:0/3177091060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 msgr2=0x55b88b88ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f039ed9d640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 0x7f03880b4f10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0398030800 tx=0x7f0398030860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f03aba16740 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0394009cb0 tx=0x7f0394011810 comp rx=0 tx=0).stop 2024-05-17T04:40:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f03aba16740 1 -- 172.21.15.50:0/3177091060 shutdown_connections 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f03aba16740 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4c34d0 0x55b88b88ec10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.183+0000 7f03aba16740 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4bb2f0 0x55b88b887cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 --2- 172.21.15.50:0/3177091060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4e41b0 0x55b88b887770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 -- 172.21.15.50:0/3177091060 >> 172.21.15.50:0/3177091060 conn(0x55b88b4d8300 msgr2=0x55b88b8831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 -- 172.21.15.50:0/3177091060 shutdown_connections 2024-05-17T04:40:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 -- 172.21.15.50:0/3177091060 wait complete. 2024-05-17T04:40:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 -- start start 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.184+0000 7f03aba16740 1 Processor -- start 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 0x55b88b891540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4e41b0 0x55b88b8984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f039e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b88b725ab0 con 0x55b88b4c34d0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b88b7a64d0 con 0x55b88b4e41b0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f039e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58790/0 (socket says 172.21.15.50:58790) 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b88b88f690 con 0x55b88b4bb2f0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f039e59c640 1 -- 172.21.15.50:0/1599923407 learned_addr learned my addr 172.21.15.50:0/1599923407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f039dd9b640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 0x55b88b891540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.185+0000 7f03aba16740 10 client.0 _ll_register_callbacks cb 0x55b88b7e4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039ed9d640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4e41b0 0x55b88b8984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 -- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4e41b0 msgr2=0x55b88b8984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4e41b0 0x55b88b8984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 -- 172.21.15.50:0/1599923407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 msgr2=0x55b88b891540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 0x55b88b891540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 -- 172.21.15.50:0/1599923407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88b7a7340 con 0x55b88b4bb2f0 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039dd9b640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 0x55b88b891540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f039e59c640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03900e0150 tx=0x7f03900a5aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a79bd3a5306b3c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f0385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:40:31.189 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900a93e0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f03900a7bc0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.186+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03940043d0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f03900a9780 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 10 client.25372 fetch_fsmap learned FSMap version 34 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b88b54f790 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03940043d0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f03900c40f0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 10 client.25372 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 20 client.25372 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4a2d80 0x55b88b89e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 4 client.25372 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:40:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88b7fbc10 con 0x55b88b4a2d80 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.187+0000 7f039dd9b640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4a2d80 0x55b88b89e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.188+0000 7f0385ffb640 10 client.25372 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.188+0000 7f039dd9b640 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4a2d80 0x55b88b89e830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e9062cfb88a27d server_cookie=8fda88c7692bff45 in_seq=0 out_seq=0 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f0385ffb640 1 -- 172.21.15.50:0/1599923407 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b88b7fbc10 con 0x55b88b4a2d80 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f0385ffb640 10 client.25372 handle_command_reply: tid=0 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 client.25372 shutdown 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 msgr2=0x55b88b891000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03900e0150 tx=0x7f03900a5aa0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 shutdown_connections 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4e41b0 0x55b88b8984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4a2d80 0x55b88b89e830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4c34d0 0x55b88b891540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 --2- 172.21.15.50:0/1599923407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4bb2f0 0x55b88b891000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 >> 172.21.15.50:0/1599923407 conn(0x55b88b4d8300 msgr2=0x55b88b886010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f0385ffb640 0 client.25372 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 shutdown_connections 2024-05-17T04:40:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 1 -- 172.21.15.50:0/1599923407 wait complete. 2024-05-17T04:40:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.189+0000 7f03aba16740 20 client.25372 trim_cache size 0 max 16384 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.211+0000 7f03aba16740 1 Processor -- start 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.211+0000 7f03aba16740 1 -- start start 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b882b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b4f60 con 0x55b88b4e41b0 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b27a0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f03aba16740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b05f0 con 0x55b88b4c34d0 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:43508/0 (socket says 172.21.15.50:43508) 2024-05-17T04:40:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 -- 172.21.15.50:0/1836390085 learned_addr learned my addr 172.21.15.50:0/1836390085 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039dd9b640 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b882b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 -- 172.21.15.50:0/1836390085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 msgr2=0x55b88b882b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b882b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 -- 172.21.15.50:0/1836390085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 msgr2=0x55b88b8830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.212+0000 7f039e59c640 1 -- 172.21.15.50:0/1836390085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88b5c8620 con 0x55b88b4bb2f0 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.213+0000 7f039e59c640 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f03900a6e50 tx=0x7f0390000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f78d3eced991b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.213+0000 7f0387fff640 1 -- 172.21.15.50:0/1836390085 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900c9070 con 0x55b88b4bb2f0 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.213+0000 7f0387fff640 1 -- 172.21.15.50:0/1836390085 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03900aa370 con 0x55b88b4bb2f0 2024-05-17T04:40:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.213+0000 7f0387fff640 1 -- 172.21.15.50:0/1836390085 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900a9c70 con 0x55b88b4bb2f0 2024-05-17T04:40:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 -- 172.21.15.50:0/1836390085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 msgr2=0x55b88b887710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f03900a6e50 tx=0x7f0390000bc0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 -- 172.21.15.50:0/1836390085 shutdown_connections 2024-05-17T04:40:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b882b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 --2- 172.21.15.50:0/1836390085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b887710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 -- 172.21.15.50:0/1836390085 >> 172.21.15.50:0/1836390085 conn(0x55b88b4d8300 msgr2=0x55b88b87df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 -- 172.21.15.50:0/1836390085 shutdown_connections 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.214+0000 7f03aba16740 1 -- 172.21.15.50:0/1836390085 wait complete. 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 -- start start 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 Processor -- start 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b897610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b897b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b0c60 con 0x55b88b4e41b0 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b20d0 con 0x55b88b4bb2f0 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.215+0000 7f03aba16740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01b1330 con 0x55b88b4c34d0 2024-05-17T04:40:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f03aba16740 10 client.0 _ll_register_callbacks cb 0x7f03900e8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b897610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b897610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:43512/0 (socket says 172.21.15.50:43512) 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039e59c640 1 -- 172.21.15.50:0/684097074 learned_addr learned my addr 172.21.15.50:0/684097074 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039dd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b897b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 -- 172.21.15.50:0/684097074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 msgr2=0x55b88b897b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b897b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 -- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 msgr2=0x55b88b897610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b897610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.216+0000 7f039ed9d640 1 -- 172.21.15.50:0/684097074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88b894a80 con 0x55b88b4e41b0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f039ed9d640 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8942f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f039802abe0 tx=0x7f039802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce13e8b3c1d88242 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f0385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398036040 con 0x55b88b4e41b0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0398004040 con 0x55b88b4e41b0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03900aa370 con 0x55b88b4e41b0 2024-05-17T04:40:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0398004240 con 0x55b88b4e41b0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.217+0000 7f03aba16740 10 client.15627 fetch_fsmap learned FSMap version 34 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b88b905a40 con 0x55b88b4e41b0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03900aa370 con 0x55b88b4e41b0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f039803c040 con 0x55b88b4e41b0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 10 client.15627 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 20 client.15627 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4781c0 0x55b88b9084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 4 client.15627 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b88b7f53f0 con 0x55b88b4781c0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.218+0000 7f039e59c640 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4781c0 0x55b88b9084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.219+0000 7f0385ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.219+0000 7f039e59c640 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4781c0 0x55b88b9084c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5225d4731aa79d1a server_cookie=19f5ab5c37b43a85 in_seq=0 out_seq=0 2024-05-17T04:40:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f0385ffb640 1 -- 172.21.15.50:0/684097074 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b88b7f53f0 con 0x55b88b4781c0 2024-05-17T04:40:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f0385ffb640 10 client.15627 handle_command_reply: tid=0 2024-05-17T04:40:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 client.15627 shutdown 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 msgr2=0x55b88b8942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8942f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f039802abe0 tx=0x7f039802a6c0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 shutdown_connections 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55b88b4781c0 0x55b88b9084c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b88b4e41b0 0x55b88b8942f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b88b4c34d0 0x55b88b897b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f0385ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 --2- 172.21.15.50:0/684097074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b88b4bb2f0 0x55b88b897610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.243+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 >> 172.21.15.50:0/684097074 conn(0x55b88b4d8300 msgr2=0x55b88b885ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.244+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 shutdown_connections 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.244+0000 7f03aba16740 1 -- 172.21.15.50:0/684097074 wait complete. 2024-05-17T04:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.244+0000 7f03aba16740 20 client.15627 trim_cache size 0 max 16384 2024-05-17T04:40:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:40:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:40:31.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 msgr2=0x7f03880b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 0x7f03880b4f10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0398030800 tx=0x7f0398030860 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 msgr2=0x7f03a0197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f039000e9c0 tx=0x7f039000edc0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 shutdown_connections 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03a0108f30 0x7f03a01a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f03880b2a00 0x7f03880b4f10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03a01065d0 0x7f03a0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 --2- 172.21.15.50:0/4060083813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03a0103c70 0x7f03a0197590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 >> 172.21.15.50:0/4060083813 conn(0x7f03a00fd880 msgr2=0x7f03a0101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 shutdown_connections 2024-05-17T04:40:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:31.245+0000 7f03a4a8c640 1 -- 172.21.15.50:0/4060083813 wait complete. 2024-05-17T04:40:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:40:31.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:30 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:40:31.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:40:31.318 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:30 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:40:31.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:40:31.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:40:31.327 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:30 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:40:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:31 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:40:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:31 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:40:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:31 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:40:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:31 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:40:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:31 smithi105 ceph-mon[34795]: pgmap v565: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-05-17T04:40:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:31 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:40:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:31 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:40:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:31 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:40:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:31 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:40:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:31 smithi110 ceph-mon[35497]: pgmap v565: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-05-17T04:40:32.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:31 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:40:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:31 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:40:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:31 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:40:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:31 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:40:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:31 smithi050 ceph-mon[31739]: pgmap v565: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 308 op/s 2024-05-17T04:40:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 95.0 s, 322.0 tps, lat 318.019 ms stddev 174.603 2024-05-17T04:40:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:33 smithi105 ceph-mon[34795]: pgmap v566: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:40:34.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:33 smithi110 ceph-mon[35497]: pgmap v566: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:40:34.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:33 smithi050 ceph-mon[31739]: pgmap v566: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:40:35.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:34 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:35.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:34 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:35.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:34 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:35 smithi105 ceph-mon[34795]: pgmap v567: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-05-17T04:40:36.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:35 smithi110 ceph-mon[35497]: pgmap v567: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-05-17T04:40:36.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:35 smithi050 ceph-mon[31739]: pgmap v567: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 286 op/s 2024-05-17T04:40:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 100.0 s, 112.0 tps, lat 878.428 ms stddev 187.372 2024-05-17T04:40:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:37 smithi105 ceph-mon[34795]: pgmap v568: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.9 MiB/s wr, 278 op/s 2024-05-17T04:40:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:37 smithi110 ceph-mon[35497]: pgmap v568: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.9 MiB/s wr, 278 op/s 2024-05-17T04:40:38.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:37 smithi050 ceph-mon[31739]: pgmap v568: 129 pgs: 129 active+clean; 938 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.9 MiB/s wr, 278 op/s 2024-05-17T04:40:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:39 smithi105 ceph-mon[34795]: pgmap v569: 129 pgs: 129 active+clean; 939 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 355 op/s 2024-05-17T04:40:40.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:39 smithi110 ceph-mon[35497]: pgmap v569: 129 pgs: 129 active+clean; 939 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 355 op/s 2024-05-17T04:40:40.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:39 smithi050 ceph-mon[31739]: pgmap v569: 129 pgs: 129 active+clean; 939 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 355 op/s 2024-05-17T04:40:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:41 smithi105 ceph-mon[34795]: pgmap v570: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-05-17T04:40:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:41 smithi110 ceph-mon[35497]: pgmap v570: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-05-17T04:40:42.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:41 smithi050 ceph-mon[31739]: pgmap v570: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-05-17T04:40:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 105.0 s, 149.6 tps, lat 671.039 ms stddev 230.936 2024-05-17T04:40:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:43 smithi105 ceph-mon[34795]: pgmap v571: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-05-17T04:40:44.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:43 smithi110 ceph-mon[35497]: pgmap v571: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-05-17T04:40:44.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:43 smithi050 ceph-mon[31739]: pgmap v571: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-05-17T04:40:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 -- 172.21.15.50:0/886821827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 msgr2=0x7f3de8101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 --2- 172.21.15.50:0/886821827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de8101e80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd802a410 comp rx=0 tx=0).stop 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 -- 172.21.15.50:0/886821827 shutdown_connections 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 --2- 172.21.15.50:0/886821827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de8102d10 0x7f3de8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 --2- 172.21.15.50:0/886821827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de81023c0 0x7f3de81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 --2- 172.21.15.50:0/886821827 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de8101e80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 -- 172.21.15.50:0/886821827 >> 172.21.15.50:0/886821827 conn(0x7f3de80fd8b0 msgr2=0x7f3de80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 -- 172.21.15.50:0/886821827 shutdown_connections 2024-05-17T04:40:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.375+0000 7f3deca8f640 1 -- 172.21.15.50:0/886821827 wait complete. 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.376+0000 7f3deca8f640 1 Processor -- start 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.376+0000 7f3deca8f640 1 -- start start 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.376+0000 7f3deca8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3deca8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3deca8f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de8102d10 0x7f3de81a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3deca8f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8110d60 con 0x7f3de8101a70 2024-05-17T04:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3deca8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8110be0 con 0x7f3de81023c0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3deca8f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8110ee0 con 0x7f3de8102d10 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54688/0 (socket says 172.21.15.50:54688) 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de8102d10 0x7f3de81a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1640397900 learned_addr learned my addr 172.21.15.50:0/1640397900 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1640397900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de8102d10 msgr2=0x7f3de81a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de8102d10 0x7f3de81a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 msgr2=0x7f3de819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1640397900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de81a3550 con 0x7f3de81023c0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de659c640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.377+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e9f0 tx=0x7f3ddc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.378+0000 7f3dd77fe640 1 -- 172.21.15.50:0/1640397900 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00cd80 con 0x7f3de81023c0 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.378+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de81a3840 con 0x7f3de81023c0 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.378+0000 7f3dd77fe640 1 -- 172.21.15.50:0/1640397900 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ddc0054b0 con 0x7f3de81023c0 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.378+0000 7f3dd77fe640 1 -- 172.21.15.50:0/1640397900 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc010430 con 0x7f3de81023c0 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.378+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de81a3d40 con 0x7f3de81023c0 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 Processor -- start 2024-05-17T04:40:44.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 -- start start 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c38931a1b0 0x55c3896bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f94d0 0x55c3896c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c38962fcc0 con 0x55c3892f94d0 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c38961f860 con 0x55c3892f12f0 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3df3a24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c3896238a0 con 0x55c38931a1b0 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3dd77fe640 1 -- 172.21.15.50:0/1640397900 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3ddc0105f0 con 0x7f3de81023c0 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3de659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f94d0 0x55c3896c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3de659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54700/0 (socket says 172.21.15.50:54700) 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3de659c640 1 -- 172.21.15.50:0/293028613 learned_addr learned my addr 172.21.15.50:0/293028613 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.381+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c38931a1b0 0x55c3896bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 -- 172.21.15.50:0/293028613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c38931a1b0 msgr2=0x55c3896bd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c38931a1b0 0x55c3896bd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 -- 172.21.15.50:0/293028613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f94d0 msgr2=0x55c3896c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f94d0 0x55c3896c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 -- 172.21.15.50:0/293028613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3895e7d50 con 0x55c3892f12f0 2024-05-17T04:40:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3de659c640 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d308006a1b5c6b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3dd4ff9640 1 -- 172.21.15.50:0/293028613 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8035030 con 0x55c3892f12f0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3dd4ff9640 1 -- 172.21.15.50:0/293028613 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dd8040070 con 0x55c3892f12f0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.382+0000 7f3dd4ff9640 1 -- 172.21.15.50:0/293028613 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8003b00 con 0x55c3892f12f0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.383+0000 7f3dd77fe640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 0x7f3dd00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.383+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 0x7f3dd00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.383+0000 7f3dd77fe640 1 -- 172.21.15.50:0/1640397900 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3ddc0e2930 con 0x7f3de81023c0 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- 172.21.15.50:0/293028613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 msgr2=0x55c3896bdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd8005d50 comp rx=0 tx=0).stop 2024-05-17T04:40:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 0x7f3dd00b4cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c3896be960 tx=0x7f3de0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- 172.21.15.50:0/293028613 shutdown_connections 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f94d0 0x55c3896c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f12f0 0x55c3896bdcf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 --2- 172.21.15.50:0/293028613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c38931a1b0 0x55c3896bd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- 172.21.15.50:0/293028613 >> 172.21.15.50:0/293028613 conn(0x55c38930e300 msgr2=0x55c3896b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- 172.21.15.50:0/293028613 shutdown_connections 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- 172.21.15.50:0/293028613 wait complete. 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 -- start start 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.384+0000 7f3df3a24740 1 Processor -- start 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c38955bab0 con 0x55c38931a1b0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c3895dc4d0 con 0x55c3892f94d0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c3896c5630 con 0x55c3892f12f0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3df3a24740 10 client.0 _ll_register_callbacks cb 0x55c38961a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42102/0 (socket says 172.21.15.50:42102) 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 -- 172.21.15.50:0/4208540085 learned_addr learned my addr 172.21.15.50:0/4208540085 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de659c640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 -- 172.21.15.50:0/4208540085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 msgr2=0x55c3896c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 -- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 msgr2=0x55c3896c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 -- 172.21.15.50:0/4208540085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3895dd340 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.385+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009cd0 tx=0x7f3de0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f512f87d8cc2027 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de000f830 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dd8040070 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3de000f9d0 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3de000fbd0 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3df3a24740 10 client.15639 fetch_fsmap learned FSMap version 36 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c389385790 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dd8040070 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.386+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f3de0026070 con 0x55c38931a1b0 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 10 client.15639 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T04:40:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 20 client.15639 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892d8d80 0x55c3896d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 4 client.15639 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:40:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c389631c10 con 0x55c3892d8d80 2024-05-17T04:40:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.387+0000 7f3de659c640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892d8d80 0x55c3896d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.388+0000 7f3dcdffb640 10 client.15639 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.388+0000 7f3de659c640 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892d8d80 0x55c3896d48e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7045280e577ebb19 server_cookie=c3793e9d8c1a0b06 in_seq=0 out_seq=0 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3dcdffb640 1 -- 172.21.15.50:0/4208540085 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c389631c10 con 0x55c3892d8d80 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3dcdffb640 10 client.15639 handle_command_reply: tid=0 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 client.15639 shutdown 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 msgr2=0x55c3896ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009cd0 tx=0x7f3de0004ab0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 shutdown_connections 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892d8d80 0x55c3896d48e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ce440 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 --2- 172.21.15.50:0/4208540085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 >> 172.21.15.50:0/4208540085 conn(0x55c38930e300 msgr2=0x55c3896bc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3dcdffb640 0 client.15639 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.389+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 shutdown_connections 2024-05-17T04:40:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.390+0000 7f3df3a24740 1 -- 172.21.15.50:0/4208540085 wait complete. 2024-05-17T04:40:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.390+0000 7f3df3a24740 20 client.15639 trim_cache size 0 max 16384 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.410+0000 7f3df3a24740 1 Processor -- start 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.410+0000 7f3df3a24740 1 -- start start 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.410+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 0x55c3896b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.410+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 0x55c3896b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 0x55c3896b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56704/0 (socket says 172.21.15.50:56704) 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 0x55c3896b9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42108/0 (socket says 172.21.15.50:42108) 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 0x55c3896b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 0x55c3896b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54720/0 (socket says 172.21.15.50:54720) 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1305455901 learned_addr learned my addr 172.21.15.50:0/1305455901 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81b0ac0 con 0x55c3892f12f0 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81ae330 con 0x55c38931a1b0 2024-05-17T04:40:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.411+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81ac180 con 0x55c3892f94d0 2024-05-17T04:40:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1305455901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 msgr2=0x55c3896b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 0x55c3896b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1305455901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 msgr2=0x55c3896b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 0x55c3896b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 -- 172.21.15.50:0/1305455901 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3893fe620 con 0x55c3892f94d0 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0a5c40 tx=0x7f3ddc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768783769b736594 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3dcffff640 1 -- 172.21.15.50:0/1305455901 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a9990 con 0x55c3892f94d0 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3dcffff640 1 -- 172.21.15.50:0/1305455901 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ddc0b2360 con 0x55c3892f94d0 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.412+0000 7f3dcffff640 1 -- 172.21.15.50:0/1305455901 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a5350 con 0x55c3892f94d0 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 msgr2=0x55c3896ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0a5c40 tx=0x7f3ddc009310 comp rx=0 tx=0).stop 2024-05-17T04:40:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 shutdown_connections 2024-05-17T04:40:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c38931a1b0 0x55c3896b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f94d0 0x55c3896ce3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 --2- 172.21.15.50:0/1305455901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c3892f12f0 0x55c3896b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 >> 172.21.15.50:0/1305455901 conn(0x55c38930e300 msgr2=0x55c3896b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 shutdown_connections 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- 172.21.15.50:0/1305455901 wait complete. 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.414+0000 7f3df3a24740 1 -- start start 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 Processor -- start 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56706/0 (socket says 172.21.15.50:56706) 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54724/0 (socket says 172.21.15.50:54724) 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 -- 172.21.15.50:0/2379493260 learned_addr learned my addr 172.21.15.50:0/2379493260 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81ac7f0 con 0x55c38931a1b0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81adc60 con 0x55c3892f94d0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3de81acec0 con 0x55c3892f12f0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de6d9d640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3df3a24740 10 client.0 _ll_register_callbacks cb 0x7f3de0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 -- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 msgr2=0x55c3896cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 -- 172.21.15.50:0/2379493260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 msgr2=0x55c3896ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.415+0000 7f3de659c640 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3896cd100 con 0x55c3892f12f0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3de659c640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565e2d2628551fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3dcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8035040 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c38962b3f0 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3dd8004240 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.416+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3dd8063050 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 10 client.25396 fetch_fsmap learned FSMap version 36 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c3896ce450 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c38962b3f0 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f3dd8054110 con 0x55c3892f12f0 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 10 client.25396 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 20 client.25396 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892ae1c0 0x55c38973eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 4 client.25396 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:40:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3895a5750 con 0x55c3892ae1c0 2024-05-17T04:40:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.417+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892ae1c0 0x55c38973eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.418+0000 7f3dcdffb640 10 client.25396 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.418+0000 7f3de5d9b640 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892ae1c0 0x55c38973eec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9dbe3789f21ebe server_cookie=b1b8340221ded210 in_seq=0 out_seq=0 2024-05-17T04:40:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3dcdffb640 1 -- 172.21.15.50:0/2379493260 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3895a5750 con 0x55c3892ae1c0 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3dcdffb640 10 client.25396 handle_command_reply: tid=0 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 client.25396 shutdown 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 msgr2=0x55c3896cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd802a410 comp rx=0 tx=0).stop 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 shutdown_connections 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c3892ae1c0 0x55c38973eec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c38931a1b0 0x55c3896ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3892f94d0 0x55c3896cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3dcdffb640 0 client.25396 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 --2- 172.21.15.50:0/2379493260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c3892f12f0 0x55c3896cb6a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 >> 172.21.15.50:0/2379493260 conn(0x55c38930e300 msgr2=0x55c3896c0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 shutdown_connections 2024-05-17T04:40:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 1 -- 172.21.15.50:0/2379493260 wait complete. 2024-05-17T04:40:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.433+0000 7f3df3a24740 20 client.25396 trim_cache size 0 max 16384 2024-05-17T04:40:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:40:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:40:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 msgr2=0x7f3dd00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 0x7f3dd00b4cf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c3896be960 tx=0x7f3de0009500 comp rx=0 tx=0).stop 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 msgr2=0x7f3de819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e9f0 tx=0x7f3ddc00edf0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 shutdown_connections 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3de8102d10 0x7f3de81a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3de81023c0 0x7f3de819bf50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3dd00b27e0 0x7f3dd00b4cf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 --2- 172.21.15.50:0/1640397900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3de8101a70 0x7f3de819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 >> 172.21.15.50:0/1640397900 conn(0x7f3de80fd8b0 msgr2=0x7f3de80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 shutdown_connections 2024-05-17T04:40:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:44.434+0000 7f3deca8f640 1 -- 172.21.15.50:0/1640397900 wait complete. 2024-05-17T04:40:45.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:44 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:45.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:44 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:45.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:44 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:40:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:45 smithi105 ceph-mon[34795]: pgmap v572: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:40:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:45 smithi110 ceph-mon[35497]: pgmap v572: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:40:46.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:45 smithi050 ceph-mon[31739]: pgmap v572: 129 pgs: 129 active+clean; 939 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-05-17T04:40:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 110.0 s, 177.6 tps, lat 582.095 ms stddev 215.065 2024-05-17T04:40:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:47 smithi105 ceph-mon[34795]: pgmap v573: 129 pgs: 129 active+clean; 940 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-05-17T04:40:48.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:47 smithi110 ceph-mon[35497]: pgmap v573: 129 pgs: 129 active+clean; 940 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-05-17T04:40:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:47 smithi050 ceph-mon[31739]: pgmap v573: 129 pgs: 129 active+clean; 940 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-05-17T04:40:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:49 smithi105 ceph-mon[34795]: pgmap v574: 129 pgs: 129 active+clean; 940 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-17T04:40:50.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:49 smithi110 ceph-mon[35497]: pgmap v574: 129 pgs: 129 active+clean; 940 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-17T04:40:50.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:49 smithi050 ceph-mon[31739]: pgmap v574: 129 pgs: 129 active+clean; 940 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-05-17T04:40:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:51 smithi105 ceph-mon[34795]: pgmap v575: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:40:52.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:51 smithi110 ceph-mon[35497]: pgmap v575: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:40:52.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:51 smithi050 ceph-mon[31739]: pgmap v575: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:40:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 115.0 s, 155.6 tps, lat 627.125 ms stddev 159.558 2024-05-17T04:40:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:53 smithi105 ceph-mon[34795]: pgmap v576: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-17T04:40:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:53 smithi110 ceph-mon[35497]: pgmap v576: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-17T04:40:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:53 smithi050 ceph-mon[31739]: pgmap v576: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.7 MiB/s wr, 401 op/s 2024-05-17T04:40:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:55 smithi105 ceph-mon[34795]: pgmap v577: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-17T04:40:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:55 smithi110 ceph-mon[35497]: pgmap v577: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-17T04:40:56.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:55 smithi050 ceph-mon[31739]: pgmap v577: 129 pgs: 129 active+clean; 940 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-17T04:40:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 -- 172.21.15.50:0/2589014210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 msgr2=0x7f3db0102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2589014210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db0102a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009930 tx=0x7f3da002a420 comp rx=0 tx=0).stop 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 -- 172.21.15.50:0/2589014210 shutdown_connections 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2589014210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2589014210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2589014210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db0102a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 -- 172.21.15.50:0/2589014210 >> 172.21.15.50:0/2589014210 conn(0x7f3db00fd010 msgr2=0x7f3db00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 -- 172.21.15.50:0/2589014210 shutdown_connections 2024-05-17T04:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.565+0000 7f3db4b43640 1 -- 172.21.15.50:0/2589014210 wait complete. 2024-05-17T04:40:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 Processor -- start 2024-05-17T04:40:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 -- start start 2024-05-17T04:40:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010eb90 con 0x7f3db0103b10 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010ea10 con 0x7f3db0102fe0 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3db4b43640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010ed10 con 0x7f3db0102600 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.566+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:50138/0 (socket says 172.21.15.50:50138) 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54154/0 (socket says 172.21.15.50:54154) 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38024/0 (socket says 172.21.15.50:38024) 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3dae59c640 1 -- 172.21.15.50:0/2105585353 learned_addr learned my addr 172.21.15.50:0/2105585353 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 -- 172.21.15.50:0/2105585353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 msgr2=0x7f3db010c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 -- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 msgr2=0x7f3db010b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 -- 172.21.15.50:0/2105585353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db010bc90 con 0x7f3db0102fe0 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3daed9d640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3dae59c640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9c0 tx=0x7f3d9800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da77fe640 1 -- 172.21.15.50:0/2105585353 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800ccc0 con 0x7f3db0102fe0 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da77fe640 1 -- 172.21.15.50:0/2105585353 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980108c0 con 0x7f3db0102fe0 2024-05-17T04:40:57.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db01a3990 con 0x7f3db0102fe0 2024-05-17T04:40:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.567+0000 7f3da77fe640 1 -- 172.21.15.50:0/2105585353 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980187a0 con 0x7f3db0102fe0 2024-05-17T04:40:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.568+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db01a3e50 con 0x7f3db0102fe0 2024-05-17T04:40:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.570+0000 7f3dbba01740 1 Processor -- start 2024-05-17T04:40:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 -- start start 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3da77fe640 1 -- 172.21.15.50:0/2105585353 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d98010a70 con 0x7f3db0102fe0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48cf2f0 0x5595b4c9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4ca2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595b4c0dcc0 con 0x5595b48d74d0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5595b4bfd860 con 0x5595b48f81b0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dbba01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5595b4c018a0 con 0x5595b48cf2f0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54160/0 (socket says 172.21.15.50:54160) 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48cf2f0 0x5595b4c9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3daed9d640 1 -- 172.21.15.50:0/2261655644 learned_addr learned my addr 172.21.15.50:0/2261655644 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4ca2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.571+0000 7f3daed9d640 1 -- 172.21.15.50:0/2261655644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48cf2f0 msgr2=0x5595b4c9bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3daed9d640 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48cf2f0 0x5595b4c9bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3daed9d640 1 -- 172.21.15.50:0/2261655644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 msgr2=0x5595b4ca2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3daed9d640 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4ca2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3daed9d640 1 -- 172.21.15.50:0/2261655644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b4bc5d50 con 0x5595b48f81b0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4ca2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3daed9d640 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3da8011810 tx=0x7f3da8011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d254d07933359636 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da4ff9640 1 -- 172.21.15.50:0/2261655644 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8011310 con 0x5595b48f81b0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da77fe640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 0x7f3d9c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da4ff9640 1 -- 172.21.15.50:0/2261655644 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3da800f9a0 con 0x5595b48f81b0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da4ff9640 1 -- 172.21.15.50:0/2261655644 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8005660 con 0x5595b48f81b0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3dae59c640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 0x7f3d9c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.572+0000 7f3da77fe640 1 -- 172.21.15.50:0/2105585353 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3d980e27e0 con 0x7f3db0102fe0 2024-05-17T04:40:57.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 -- 172.21.15.50:0/2261655644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 msgr2=0x5595b4c9b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dae59c640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 0x7f3d9c0b4cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3da0000c00 tx=0x7f3da0031910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3da8011810 tx=0x7f3da8011cd0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 -- 172.21.15.50:0/2261655644 shutdown_connections 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4ca2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48cf2f0 0x5595b4c9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2261655644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48f81b0 0x5595b4c9b770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 -- 172.21.15.50:0/2261655644 >> 172.21.15.50:0/2261655644 conn(0x5595b48ec300 msgr2=0x5595b4c971a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 -- 172.21.15.50:0/2261655644 shutdown_connections 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.573+0000 7f3dbba01740 1 -- 172.21.15.50:0/2261655644 wait complete. 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 -- start start 2024-05-17T04:40:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 Processor -- start 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4ca5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4ca56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595b4b39ab0 con 0x5595b48cf2f0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5595b4bba4d0 con 0x5595b48d74d0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5595b4ca3690 con 0x5595b48f81b0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dbba01740 10 client.0 _ll_register_callbacks cb 0x5595b4bf8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4ca5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4ca56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4ca5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38040/0 (socket says 172.21.15.50:38040) 2024-05-17T04:40:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3dae59c640 1 -- 172.21.15.50:0/1226386811 learned_addr learned my addr 172.21.15.50:0/1226386811 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.574+0000 7f3daed9d640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 -- 172.21.15.50:0/1226386811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 msgr2=0x5595b4cac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 -- 172.21.15.50:0/1226386811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 msgr2=0x5595b4ca5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4ca5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 -- 172.21.15.50:0/1226386811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b4bbb340 con 0x5595b48d74d0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3da7fff640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4ca56c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d980dffa0 tx=0x7f3d980a9480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6894c6965ca1dce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980a9580 con 0x5595b48d74d0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3d98010640 con 0x5595b48d74d0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da800f9a0 con 0x5595b48d74d0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3d980aa170 con 0x5595b48d74d0 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3dbba01740 10 client.25415 fetch_fsmap learned FSMap version 36 2024-05-17T04:40:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.575+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595b4963790 con 0x5595b48d74d0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3da800f9a0 con 0x5595b48d74d0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f3d980a8d60 con 0x5595b48d74d0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 10 client.25415 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 20 client.25415 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b48b6d80 0x5595b4cbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 4 client.25415 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595b4c0fc10 con 0x5595b48b6d80 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.576+0000 7f3dae59c640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b48b6d80 0x5595b4cbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.577+0000 7f3d95ffb640 10 client.25415 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.577+0000 7f3dae59c640 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b48b6d80 0x5595b4cbb0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4013352b59c3a7c0 server_cookie=188f97fd1448c0fa in_seq=0 out_seq=0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3d95ffb640 1 -- 172.21.15.50:0/1226386811 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595b4c0fc10 con 0x5595b48b6d80 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3d95ffb640 10 client.25415 handle_command_reply: tid=0 2024-05-17T04:40:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 client.25415 shutdown 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 msgr2=0x5595b4ca56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4ca56c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d980dffa0 tx=0x7f3d980a9480 comp rx=0 tx=0).stop 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 shutdown_connections 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4ca56c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b48b6d80 0x5595b4cbb0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 --2- 172.21.15.50:0/1226386811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4ca5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3d95ffb640 0 client.25415 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 >> 172.21.15.50:0/1226386811 conn(0x5595b48ec300 msgr2=0x5595b4c9a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 shutdown_connections 2024-05-17T04:40:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 1 -- 172.21.15.50:0/1226386811 wait complete. 2024-05-17T04:40:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.578+0000 7f3dbba01740 20 client.25415 trim_cache size 0 max 16384 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.599+0000 7f3dbba01740 1 Processor -- start 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.599+0000 7f3dbba01740 1 -- start start 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48cf2f0 0x5595b4c9fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cbfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01ac770 con 0x5595b48d74d0 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a9fe0 con 0x5595b48cf2f0 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dbba01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a7e30 con 0x5595b48f81b0 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38054/0 (socket says 172.21.15.50:38054) 2024-05-17T04:40:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 -- 172.21.15.50:0/2040503907 learned_addr learned my addr 172.21.15.50:0/2040503907 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3daed9d640 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cbfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3dae59c640 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48cf2f0 0x5595b4c9fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 -- 172.21.15.50:0/2040503907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 msgr2=0x5595b4cbfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cbfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 -- 172.21.15.50:0/2040503907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48cf2f0 msgr2=0x5595b4c9fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48cf2f0 0x5595b4c9fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.600+0000 7f3da7fff640 1 -- 172.21.15.50:0/2040503907 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595b49dc620 con 0x5595b48d74d0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.601+0000 7f3da7fff640 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d980a9770 tx=0x7f3d980e2b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d2294931e3b1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.601+0000 7f3d97fff640 1 -- 172.21.15.50:0/2040503907 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980a9580 con 0x5595b48d74d0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.601+0000 7f3d97fff640 1 -- 172.21.15.50:0/2040503907 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980ade20 con 0x5595b48d74d0 2024-05-17T04:40:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.601+0000 7f3d97fff640 1 -- 172.21.15.50:0/2040503907 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980a8cd0 con 0x5595b48d74d0 2024-05-17T04:40:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- 172.21.15.50:0/2040503907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 msgr2=0x5595b4cbf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3d980a9770 tx=0x7f3d980e2b50 comp rx=0 tx=0).stop 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- 172.21.15.50:0/2040503907 shutdown_connections 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4cbfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48d74d0 0x5595b4cbf860 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 --2- 172.21.15.50:0/2040503907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48cf2f0 0x5595b4c9fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- 172.21.15.50:0/2040503907 >> 172.21.15.50:0/2040503907 conn(0x5595b48ec300 msgr2=0x5595b4c9a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- 172.21.15.50:0/2040503907 shutdown_connections 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- 172.21.15.50:0/2040503907 wait complete. 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 -- start start 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.602+0000 7f3dbba01740 1 Processor -- start 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4cab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4d215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a84a0 con 0x5595b48cf2f0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a9910 con 0x5595b48d74d0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a8b70 con 0x5595b48f81b0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dbba01740 10 client.0 _ll_register_callbacks cb 0x7f3d980a9840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4d215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38058/0 (socket says 172.21.15.50:38058) 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3dae59c640 1 -- 172.21.15.50:0/4000388702 learned_addr learned my addr 172.21.15.50:0/4000388702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3da7fff640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4cab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 -- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 msgr2=0x5595b4cab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4cab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 -- 172.21.15.50:0/4000388702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 msgr2=0x5595b4caaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.603+0000 7f3daed9d640 1 -- 172.21.15.50:0/4000388702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595b4d21bb0 con 0x5595b48f81b0 2024-05-17T04:40:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3dae59c640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3daed9d640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4d215c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006010 tx=0x7f3da8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca080b8884763b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800f9a0 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3da801e030 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d980ade20 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3da8036050 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3dbba01740 10 client.25414 fetch_fsmap learned FSMap version 36 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.604+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595b4d22e40 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d980ade20 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f3da8024040 con 0x5595b48f81b0 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 10 client.25414 fetch_fsmap finished waiting for FSMap version 36 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 10 client.25414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 20 client.25414 populate_metadata read hostname 'smithi050' 2024-05-17T04:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b488c1c0 0x5595b4d25690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:40:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 4 client.25414 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:40:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595b4c093f0 con 0x5595b488c1c0 2024-05-17T04:40:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.605+0000 7f3dae59c640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b488c1c0 0x5595b4d25690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:40:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.606+0000 7f3d95ffb640 10 client.25414 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.606+0000 7f3dae59c640 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b488c1c0 0x5595b4d25690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6af4c19e2ac54d server_cookie=549b985313edde60 in_seq=0 out_seq=0 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.627+0000 7f3d95ffb640 1 -- 172.21.15.50:0/4000388702 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5595b4c093f0 con 0x5595b488c1c0 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.627+0000 7f3d95ffb640 10 client.25414 handle_command_reply: tid=0 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.627+0000 7f3dbba01740 1 client.25414 shutdown 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 msgr2=0x5595b4d215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4d215c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006010 tx=0x7f3da8003ba0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 shutdown_connections 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595b48f81b0 0x5595b4d215c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595b48d74d0 0x5595b4cab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5595b488c1c0 0x5595b4d25690 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3d95ffb640 0 client.25414 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 --2- 172.21.15.50:0/4000388702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595b48cf2f0 0x5595b4caaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 >> 172.21.15.50:0/4000388702 conn(0x5595b48ec300 msgr2=0x5595b4c9ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 shutdown_connections 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 1 -- 172.21.15.50:0/4000388702 wait complete. 2024-05-17T04:40:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.628+0000 7f3dbba01740 20 client.25414 trim_cache size 0 max 16384 2024-05-17T04:40:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:40:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:40:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:40:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.629+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 msgr2=0x7f3d9c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.629+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 0x7f3d9c0b4cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3da0000c00 tx=0x7f3da0031910 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.629+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 msgr2=0x7f3db010b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.629+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9c0 tx=0x7f3d9800edc0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 shutdown_connections 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3d9c0b27e0 0x7f3d9c0b4cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db0103b10 0x7f3db010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3db0102fe0 0x7f3db010b020 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 --2- 172.21.15.50:0/2105585353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3db0102600 0x7f3db010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 >> 172.21.15.50:0/2105585353 conn(0x7f3db00fd010 msgr2=0x7f3db01066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 shutdown_connections 2024-05-17T04:40:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:40:57.630+0000 7f3db4b43640 1 -- 172.21.15.50:0/2105585353 wait complete. 2024-05-17T04:40:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 120.0 s, 151.8 tps, lat 658.238 ms stddev 153.060 2024-05-17T04:40:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:57 smithi105 ceph-mon[34795]: pgmap v578: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.2 MiB/s wr, 361 op/s 2024-05-17T04:40:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:57 smithi110 ceph-mon[35497]: pgmap v578: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.2 MiB/s wr, 361 op/s 2024-05-17T04:40:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:57 smithi050 ceph-mon[31739]: pgmap v578: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.2 MiB/s wr, 361 op/s 2024-05-17T04:41:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:40:59 smithi105 ceph-mon[34795]: pgmap v579: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.5 MiB/s wr, 369 op/s 2024-05-17T04:41:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:40:59 smithi110 ceph-mon[35497]: pgmap v579: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.5 MiB/s wr, 369 op/s 2024-05-17T04:41:00.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:40:59 smithi050 ceph-mon[31739]: pgmap v579: 129 pgs: 129 active+clean; 941 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.5 MiB/s wr, 369 op/s 2024-05-17T04:41:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:01 smithi105 ceph-mon[34795]: pgmap v580: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-05-17T04:41:02.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:01 smithi110 ceph-mon[35497]: pgmap v580: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-05-17T04:41:02.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:01 smithi050 ceph-mon[31739]: pgmap v580: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.2 MiB/s wr, 362 op/s 2024-05-17T04:41:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 125.0 s, 132.2 tps, lat 753.967 ms stddev 178.233 2024-05-17T04:41:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:03 smithi105 ceph-mon[34795]: pgmap v581: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:41:04.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:03 smithi110 ceph-mon[35497]: pgmap v581: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:41:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:03 smithi050 ceph-mon[31739]: pgmap v581: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:41:05.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:04 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:41:05.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:04 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:41:05.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:04 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:41:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:05 smithi105 ceph-mon[34795]: pgmap v582: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:41:06.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:05 smithi110 ceph-mon[35497]: pgmap v582: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:41:06.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:05 smithi050 ceph-mon[31739]: pgmap v582: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:41:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 130.0 s, 132.6 tps, lat 749.969 ms stddev 170.544 2024-05-17T04:41:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:07 smithi105 ceph-mon[34795]: pgmap v583: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:07 smithi110 ceph-mon[35497]: pgmap v583: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:07 smithi050 ceph-mon[31739]: pgmap v583: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:10 smithi105 ceph-mon[34795]: pgmap v584: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-05-17T04:41:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:10 smithi110 ceph-mon[35497]: pgmap v584: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-05-17T04:41:10.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:10 smithi050 ceph-mon[31739]: pgmap v584: 129 pgs: 129 active+clean; 941 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-05-17T04:41:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 -- 172.21.15.50:0/2222044118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90741065e0 msgr2=0x7f9074108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 --2- 172.21.15.50:0/2222044118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90741065e0 0x7f9074108a00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9068009900 tx=0x7f906802a410 comp rx=0 tx=0).stop 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 -- 172.21.15.50:0/2222044118 shutdown_connections 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 --2- 172.21.15.50:0/2222044118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f907410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 --2- 172.21.15.50:0/2222044118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90741065e0 0x7f9074108a00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 --2- 172.21.15.50:0/2222044118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9074103c80 0x7f90741060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 -- 172.21.15.50:0/2222044118 >> 172.21.15.50:0/2222044118 conn(0x7f90740fd8b0 msgr2=0x7f90740ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 -- 172.21.15.50:0/2222044118 shutdown_connections 2024-05-17T04:41:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.760+0000 7f9078fcb640 1 -- 172.21.15.50:0/2222044118 wait complete. 2024-05-17T04:41:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.761+0000 7f9078fcb640 1 Processor -- start 2024-05-17T04:41:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.761+0000 7f9078fcb640 1 -- start start 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 0x7f9074199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90741065e0 0x7f9074199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f90741a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f907410eb60 con 0x7f9074103c80 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f907410e9e0 con 0x7f9074108f40 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9078fcb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f907410ece0 con 0x7f90741065e0 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f907259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 0x7f9074199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f907259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 0x7f9074199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44070/0 (socket says 172.21.15.50:44070) 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f907259c640 1 -- 172.21.15.50:0/288823497 learned_addr learned my addr 172.21.15.50:0/288823497 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f90741a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9071d9b640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90741065e0 0x7f9074199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 -- 172.21.15.50:0/288823497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90741065e0 msgr2=0x7f9074199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90741065e0 0x7f9074199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 -- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 msgr2=0x7f9074199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 0x7f9074199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 -- 172.21.15.50:0/288823497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741a1360 con 0x7f9074108f40 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.762+0000 7f9072d9d640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f90741a0cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.763+0000 7f90637fe640 1 -- 172.21.15.50:0/288823497 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00ce20 con 0x7f9074108f40 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.763+0000 7f90637fe640 1 -- 172.21.15.50:0/288823497 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f906c005020 con 0x7f9074108f40 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.763+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741a1650 con 0x7f9074108f40 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.763+0000 7f90637fe640 1 -- 172.21.15.50:0/288823497 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c010690 con 0x7f9074108f40 2024-05-17T04:41:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.763+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a1b50 con 0x7f9074108f40 2024-05-17T04:41:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.766+0000 7f907fb22740 1 Processor -- start 2024-05-17T04:41:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.766+0000 7f90637fe640 1 -- 172.21.15.50:0/288823497 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f906c010830 con 0x7f9074108f40 2024-05-17T04:41:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 -- start start 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fb081b0 0x55fa7feab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 0x55fa7feabcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fe1dcc0 con 0x55fa7fadf2f0 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fe0d860 con 0x55fa7fae74d0 2024-05-17T04:41:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907fb22740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fe118a0 con 0x55fa7fb081b0 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fb081b0 0x55fa7feab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:51746/0 (socket says 172.21.15.50:51746) 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f9071d9b640 1 -- 172.21.15.50:0/3465174185 learned_addr learned my addr 172.21.15.50:0/3465174185 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.767+0000 7f907259c640 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 0x55fa7feabcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 -- 172.21.15.50:0/3465174185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fb081b0 msgr2=0x55fa7feab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fb081b0 0x55fa7feab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 -- 172.21.15.50:0/3465174185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 msgr2=0x55fa7feabcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f90637fe640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 0x7f905c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 0x55fa7feabcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 -- 172.21.15.50:0/3465174185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7fdd5d50 con 0x55fa7fae74d0 2024-05-17T04:41:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f907259c640 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 0x55fa7feabcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f90637fe640 1 -- 172.21.15.50:0/288823497 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f906c0e2d10 con 0x7f9074108f40 2024-05-17T04:41:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 0x7f905c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.768+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90680096c0 tx=0x7f906802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80f6763de5c1f61a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.769+0000 7f9060ff9640 1 -- 172.21.15.50:0/3465174185 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068052070 con 0x55fa7fae74d0 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.769+0000 7f9060ff9640 1 -- 172.21.15.50:0/3465174185 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90680044c0 con 0x55fa7fae74d0 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.769+0000 7f9060ff9640 1 -- 172.21.15.50:0/3465174185 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068035070 con 0x55fa7fae74d0 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f9071d9b640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 0x7f905c0b4d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9068047000 tx=0x7f9068048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 -- 172.21.15.50:0/3465174185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 msgr2=0x55fa7feb2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f90680096c0 tx=0x7f906802a410 comp rx=0 tx=0).stop 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 -- 172.21.15.50:0/3465174185 shutdown_connections 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb2c10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fadf2f0 0x55fa7feabcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 --2- 172.21.15.50:0/3465174185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fb081b0 0x55fa7feab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.770+0000 7f907fb22740 1 -- 172.21.15.50:0/3465174185 >> 172.21.15.50:0/3465174185 conn(0x55fa7fafc300 msgr2=0x55fa7fea71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- 172.21.15.50:0/3465174185 shutdown_connections 2024-05-17T04:41:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- 172.21.15.50:0/3465174185 wait complete. 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- start start 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 Processor -- start 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fd49ab0 con 0x55fa7fb081b0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa7fdca4d0 con 0x55fa7fae74d0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.771+0000 7f907fb22740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa7feb3690 con 0x55fa7fadf2f0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f907fb22740 10 client.0 _ll_register_callbacks cb 0x55fa7fe08950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9072d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:51758/0 (socket says 172.21.15.50:51758) 2024-05-17T04:41:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9072d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44084/0 (socket says 172.21.15.50:44084) 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 -- 172.21.15.50:0/3938432138 learned_addr learned my addr 172.21.15.50:0/3938432138 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f907259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 -- 172.21.15.50:0/3938432138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 msgr2=0x55fa7feb5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 -- 172.21.15.50:0/3938432138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 msgr2=0x55fa7febc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 -- 172.21.15.50:0/3938432138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7fdcb340 con 0x55fa7fae74d0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9068031770 tx=0x7f9068005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9462f7ee897d1a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.772+0000 7f9059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906805b020 con 0x55fa7fae74d0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9068003e70 con 0x55fa7fae74d0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa7fe1fc10 con 0x55fa7fae74d0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9068035070 con 0x55fa7fae74d0 2024-05-17T04:41:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 10 client.25433 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa7fb73790 con 0x55fa7fae74d0 2024-05-17T04:41:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fa7fe1fc10 con 0x55fa7fae74d0 2024-05-17T04:41:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f906804a020 con 0x55fa7fae74d0 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 10 client.25433 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.773+0000 7f907fb22740 20 client.25433 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.774+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fac6d80 0x55fa7fec2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.774+0000 7f907fb22740 4 client.25433 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.774+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7fe193f0 con 0x55fa7fac6d80 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.774+0000 7f907259c640 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fac6d80 0x55fa7fec2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.775+0000 7f907259c640 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fac6d80 0x55fa7fec2410 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780ebb41be16aa79 server_cookie=46ee56fef76af211 in_seq=0 out_seq=0 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.775+0000 7f9059ffb640 10 client.25433 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.776+0000 7f9059ffb640 1 -- 172.21.15.50:0/3938432138 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa7fe193f0 con 0x55fa7fac6d80 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.776+0000 7f9059ffb640 10 client.25433 handle_command_reply: tid=0 2024-05-17T04:41:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.776+0000 7f907fb22740 1 client.25433 shutdown 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 msgr2=0x55fa7feb55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9068031770 tx=0x7f9068005c90 comp rx=0 tx=0).stop 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 shutdown_connections 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fac6d80 0x55fa7fec2410 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7feb55d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 --2- 172.21.15.50:0/3938432138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f9059ffb640 0 client.25433 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 >> 172.21.15.50:0/3938432138 conn(0x55fa7fafc300 msgr2=0x55fa7fea61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 shutdown_connections 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 1 -- 172.21.15.50:0/3938432138 wait complete. 2024-05-17T04:41:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.777+0000 7f907fb22740 20 client.25433 trim_cache size 0 max 16384 2024-05-17T04:41:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 Processor -- start 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 -- start start 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7febc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90741b0b30 con 0x55fa7fb081b0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ae3a0 con 0x55fa7fae74d0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907fb22740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ac1f0 con 0x55fa7fadf2f0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f9072d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7febc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:54878/0 (socket says 172.21.15.50:54878) 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f9072d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44086/0 (socket says 172.21.15.50:44086) 2024-05-17T04:41:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.797+0000 7f907259c640 1 -- 172.21.15.50:0/1142291625 learned_addr learned my addr 172.21.15.50:0/1142291625 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 -- 172.21.15.50:0/1142291625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 msgr2=0x55fa7febc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7febc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 -- 172.21.15.50:0/1142291625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 msgr2=0x55fa7febca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 -- 172.21.15.50:0/1142291625 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7fbec620 con 0x55fa7fadf2f0 2024-05-17T04:41:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f907259c640 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55fa7feacb70 tx=0x7f9064005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfa70027db6690b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f905bfff640 1 -- 172.21.15.50:0/1142291625 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064005020 con 0x55fa7fadf2f0 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f905bfff640 1 -- 172.21.15.50:0/1142291625 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90640047a0 con 0x55fa7fadf2f0 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.798+0000 7f905bfff640 1 -- 172.21.15.50:0/1142291625 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640223d0 con 0x55fa7fadf2f0 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 -- 172.21.15.50:0/1142291625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 msgr2=0x55fa7fea5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55fa7feacb70 tx=0x7f9064005f90 comp rx=0 tx=0).stop 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 -- 172.21.15.50:0/1142291625 shutdown_connections 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fb081b0 0x55fa7febca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fae74d0 0x55fa7febc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 --2- 172.21.15.50:0/1142291625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7fea5ef0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 -- 172.21.15.50:0/1142291625 >> 172.21.15.50:0/1142291625 conn(0x55fa7fafc300 msgr2=0x55fa7fea57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 -- 172.21.15.50:0/1142291625 shutdown_connections 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.799+0000 7f907fb22740 1 -- 172.21.15.50:0/1142291625 wait complete. 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 -- start start 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 Processor -- start 2024-05-17T04:41:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 0x55fa7ff28660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ac860 con 0x55fa7fae74d0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90741adcd0 con 0x55fa7fb081b0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90741acf30 con 0x55fa7fadf2f0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907fb22740 10 client.0 _ll_register_callbacks cb 0x7f9064009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f9071d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44088/0 (socket says 172.21.15.50:44088) 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f9071d9b640 1 -- 172.21.15.50:0/3946087835 learned_addr learned my addr 172.21.15.50:0/3946087835 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f907259c640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.800+0000 7f9072d9d640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 0x55fa7ff28660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 -- 172.21.15.50:0/3946087835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 msgr2=0x55fa7feb98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 -- 172.21.15.50:0/3946087835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 msgr2=0x55fa7feb9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 -- 172.21.15.50:0/3946087835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7febce00 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9071d9b640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9072d9d640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 0x55fa7ff28660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f906c0a9ed0 tx=0x7f906c0aa000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682f0514ad32a889 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0c8070 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f906c0b2910 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90640047a0 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f906c08b050 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f907fb22740 10 client.25439 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.801+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa7ff29a90 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90640047a0 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f906c0a5940 con 0x55fa7fb081b0 2024-05-17T04:41:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 10 client.25439 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 20 client.25439 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fa9c1c0 0x55fa7ff2c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 4 client.25439 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa7fd93750 con 0x55fa7fa9c1c0 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.802+0000 7f907259c640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fa9c1c0 0x55fa7ff2c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.803+0000 7f9059ffb640 10 client.25439 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.803+0000 7f907259c640 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fa9c1c0 0x55fa7ff2c550 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594458f3c747f4b2 server_cookie=89cbf59eb3a77434 in_seq=0 out_seq=0 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.824+0000 7f9059ffb640 1 -- 172.21.15.50:0/3946087835 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa7fd93750 con 0x55fa7fa9c1c0 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.824+0000 7f9059ffb640 10 client.25439 handle_command_reply: tid=0 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.824+0000 7f907fb22740 1 client.25439 shutdown 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 msgr2=0x55fa7ff28660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 0x55fa7ff28660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f906c0a9ed0 tx=0x7f906c0aa000 comp rx=0 tx=0).stop 2024-05-17T04:41:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 shutdown_connections 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa7fb081b0 0x55fa7ff28660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55fa7fa9c1c0 0x55fa7ff2c550 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fa7fae74d0 0x55fa7feb9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 --2- 172.21.15.50:0/3946087835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa7fadf2f0 0x55fa7feb98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 >> 172.21.15.50:0/3946087835 conn(0x55fa7fafc300 msgr2=0x55fa7fea9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f9059ffb640 0 client.25439 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 shutdown_connections 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 1 -- 172.21.15.50:0/3946087835 wait complete. 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.825+0000 7f907fb22740 20 client.25439 trim_cache size 0 max 16384 2024-05-17T04:41:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 msgr2=0x7f905c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 0x7f905c0b4d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9068047000 tx=0x7f9068048000 comp rx=0 tx=0).stop 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 msgr2=0x7f90741a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f90741a0cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00ef10 comp rx=0 tx=0).stop 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 shutdown_connections 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074108f40 0x7f90741a0cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90741065e0 0x7f9074199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f905c0b2860 0x7f905c0b4d70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 --2- 172.21.15.50:0/288823497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9074103c80 0x7f9074199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 >> 172.21.15.50:0/288823497 conn(0x7f90740fd8b0 msgr2=0x7f9074105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 shutdown_connections 2024-05-17T04:41:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:10.826+0000 7f9078fcb640 1 -- 172.21.15.50:0/288823497 wait complete. 2024-05-17T04:41:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:12 smithi105 ceph-mon[34795]: pgmap v585: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-17T04:41:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:12 smithi110 ceph-mon[35497]: pgmap v585: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-17T04:41:12.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:12 smithi050 ceph-mon[31739]: pgmap v585: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.1 MiB/s wr, 351 op/s 2024-05-17T04:41:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 135.0 s, 126.0 tps, lat 798.624 ms stddev 167.810 2024-05-17T04:41:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:14 smithi105 ceph-mon[34795]: pgmap v586: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 358 op/s 2024-05-17T04:41:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:14 smithi110 ceph-mon[35497]: pgmap v586: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 358 op/s 2024-05-17T04:41:14.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:14 smithi050 ceph-mon[31739]: pgmap v586: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.4 MiB/s wr, 358 op/s 2024-05-17T04:41:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:16 smithi105 ceph-mon[34795]: pgmap v587: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:41:16.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:16 smithi110 ceph-mon[35497]: pgmap v587: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:41:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:16 smithi050 ceph-mon[31739]: pgmap v587: 129 pgs: 129 active+clean; 942 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:41:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 140.0 s, 118.8 tps, lat 843.413 ms stddev 236.100 2024-05-17T04:41:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:18 smithi105 ceph-mon[34795]: pgmap v588: 129 pgs: 129 active+clean; 942 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:18.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:18 smithi110 ceph-mon[35497]: pgmap v588: 129 pgs: 129 active+clean; 942 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:18 smithi050 ceph-mon[31739]: pgmap v588: 129 pgs: 129 active+clean; 942 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-05-17T04:41:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:20 smithi105 ceph-mon[34795]: pgmap v589: 129 pgs: 129 active+clean; 943 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-05-17T04:41:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:20 smithi110 ceph-mon[35497]: pgmap v589: 129 pgs: 129 active+clean; 943 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-05-17T04:41:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:20 smithi050 ceph-mon[31739]: pgmap v589: 129 pgs: 129 active+clean; 943 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 353 op/s 2024-05-17T04:41:22.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:22 smithi050 ceph-mon[31739]: pgmap v590: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 399 op/s 2024-05-17T04:41:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:22 smithi105 ceph-mon[34795]: pgmap v590: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 399 op/s 2024-05-17T04:41:22.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:22 smithi110 ceph-mon[35497]: pgmap v590: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 399 op/s 2024-05-17T04:41:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 145.0 s, 118.2 tps, lat 839.322 ms stddev 190.077 2024-05-17T04:41:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 -- 172.21.15.50:0/157262673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff781042b0 msgr2=0x7eff781046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 --2- 172.21.15.50:0/157262673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff781042b0 0x7eff781046c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff68009900 tx=0x7eff6802a410 comp rx=0 tx=0).stop 2024-05-17T04:41:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 -- 172.21.15.50:0/157262673 shutdown_connections 2024-05-17T04:41:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 --2- 172.21.15.50:0/157262673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff78104c00 0x7eff7810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 --2- 172.21.15.50:0/157262673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff781042b0 0x7eff781046c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 --2- 172.21.15.50:0/157262673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78103190 0x7eff781035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 -- 172.21.15.50:0/157262673 >> 172.21.15.50:0/157262673 conn(0x7eff780fe970 msgr2=0x7eff78100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 -- 172.21.15.50:0/157262673 shutdown_connections 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 -- 172.21.15.50:0/157262673 wait complete. 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.960+0000 7eff7ee46640 1 Processor -- start 2024-05-17T04:41:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 -- start start 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781042b0 0x7eff78199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 0x7eff781a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff7810eb70 con 0x7eff78103190 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff7810e9f0 con 0x7eff781042b0 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7ee46640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff7810ecf0 con 0x7eff78104c00 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37306/0 (socket says 172.21.15.50:37306) 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 -- 172.21.15.50:0/408465367 learned_addr learned my addr 172.21.15.50:0/408465367 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 0x7eff781a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff6ffff640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781042b0 0x7eff78199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 -- 172.21.15.50:0/408465367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 msgr2=0x7eff781a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 0x7eff781a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 -- 172.21.15.50:0/408465367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781042b0 msgr2=0x7eff78199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781042b0 0x7eff78199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7cbe2640 1 -- 172.21.15.50:0/408465367 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff781a1350 con 0x7eff78103190 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.961+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 0x7eff781a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e9f0 tx=0x7eff7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff6dffb640 1 -- 172.21.15.50:0/408465367 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7400ccc0 con 0x7eff78103190 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff781a1640 con 0x7eff78103190 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff6dffb640 1 -- 172.21.15.50:0/408465367 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff740054b0 con 0x7eff78103190 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff6dffb640 1 -- 172.21.15.50:0/408465367 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74010430 con 0x7eff78103190 2024-05-17T04:41:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.962+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff781a1b90 con 0x7eff78103190 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.965+0000 7eff85dd8740 1 Processor -- start 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.965+0000 7eff85dd8740 1 -- start start 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 0x55578e548100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 0x55578e54f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff6dffb640 1 -- 172.21.15.50:0/408465367 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7eff740105d0 con 0x7eff78103190 2024-05-17T04:41:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55578e4b5cc0 con 0x55578e1772f0 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55578e4a5860 con 0x55578e17f4d0 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff85dd8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55578e4a98a0 con 0x55578e1a01b0 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51548/0 (socket says 172.21.15.50:51548) 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff7d3e3640 1 -- 172.21.15.50:0/3091798179 learned_addr learned my addr 172.21.15.50:0/3091798179 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff6ffff640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 0x55578e54f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.966+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 0x55578e548100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 -- 172.21.15.50:0/3091798179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 msgr2=0x55578e54f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 0x55578e54f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 -- 172.21.15.50:0/3091798179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 msgr2=0x55578e548100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 0x55578e548100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 -- 172.21.15.50:0/3091798179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578e46dd50 con 0x55578e1a01b0 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 0x55578e548100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff6ffff640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 0x55578e54f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.967+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff700117e0 tx=0x7eff70011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d615f84320884b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff6dffb640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 0x7eff600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff5effd640 1 -- 172.21.15.50:0/3091798179 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70011310 con 0x55578e1a01b0 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff6dffb640 1 -- 172.21.15.50:0/408465367 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7eff740e29a0 con 0x7eff78103190 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff5effd640 1 -- 172.21.15.50:0/3091798179 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff70003e00 con 0x55578e1a01b0 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff5effd640 1 -- 172.21.15.50:0/3091798179 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70004410 con 0x55578e1a01b0 2024-05-17T04:41:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.968+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 0x7eff600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff7cbe2640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 0x7eff600b4cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff740a6e40 tx=0x7eff7400ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 -- 172.21.15.50:0/3091798179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 msgr2=0x55578e547b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff700117e0 tx=0x7eff70011ca0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 -- 172.21.15.50:0/3091798179 shutdown_connections 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e17f4d0 0x55578e54f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1772f0 0x55578e548100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 --2- 172.21.15.50:0/3091798179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e547b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 -- 172.21.15.50:0/3091798179 >> 172.21.15.50:0/3091798179 conn(0x55578e194300 msgr2=0x55578e53f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 -- 172.21.15.50:0/3091798179 shutdown_connections 2024-05-17T04:41:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.970+0000 7eff85dd8740 1 -- 172.21.15.50:0/3091798179 wait complete. 2024-05-17T04:41:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.971+0000 7eff85dd8740 1 -- start start 2024-05-17T04:41:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.971+0000 7eff85dd8740 1 Processor -- start 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.971+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.971+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e17f4d0 0x55578e5519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:60668/0 (socket says 172.21.15.50:60668) 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 -- 172.21.15.50:0/4018440525 learned_addr learned my addr 172.21.15.50:0/4018440525 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e558940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55578e3e1ab0 con 0x55578e17f4d0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55578e4624d0 con 0x55578e1772f0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55578e54faa0 con 0x55578e1a01b0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff85dd8740 10 client.0 _ll_register_callbacks cb 0x55578e4a0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff6ffff640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e17f4d0 0x55578e5519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e558940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 -- 172.21.15.50:0/4018440525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 msgr2=0x55578e558940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e558940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 -- 172.21.15.50:0/4018440525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e17f4d0 msgr2=0x55578e5519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e17f4d0 0x55578e5519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.972+0000 7eff7d3e3640 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578e463340 con 0x55578e1772f0 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff70011610 tx=0x7eff70005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c28735ab6bb02f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff57fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:41:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff700043b0 con 0x55578e1772f0 2024-05-17T04:41:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7eff70004550 con 0x55578e1772f0 2024-05-17T04:41:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff70003e00 con 0x55578e1772f0 2024-05-17T04:41:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff70004750 con 0x55578e1772f0 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.973+0000 7eff85dd8740 10 client.25451 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55578e20b790 con 0x55578e1772f0 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff70003e00 con 0x55578e1772f0 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7eff70018020 con 0x55578e1772f0 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 10 client.25451 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 20 client.25451 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e15ed80 0x55578e55e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 4 client.25451 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578e4b7c10 con 0x55578e15ed80 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.974+0000 7eff6ffff640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e15ed80 0x55578e55e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.975+0000 7eff57fff640 10 client.25451 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.975+0000 7eff6ffff640 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e15ed80 0x55578e55e930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442ba62c8aa81ae3 server_cookie=1b956a636ea20935 in_seq=0 out_seq=0 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff57fff640 1 -- 172.21.15.50:0/4018440525 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55578e4b7c10 con 0x55578e15ed80 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff57fff640 10 client.25451 handle_command_reply: tid=0 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 client.25451 shutdown 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 msgr2=0x55578e5514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff70011610 tx=0x7eff70005fd0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 shutdown_connections 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e1a01b0 0x55578e558940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e15ed80 0x55578e55e930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e17f4d0 0x55578e5519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 --2- 172.21.15.50:0/4018440525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e5514a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 >> 172.21.15.50:0/4018440525 conn(0x55578e194300 msgr2=0x55578e542010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff57fff640 0 client.25451 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 shutdown_connections 2024-05-17T04:41:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.976+0000 7eff85dd8740 1 -- 172.21.15.50:0/4018440525 wait complete. 2024-05-17T04:41:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.977+0000 7eff85dd8740 20 client.25451 trim_cache size 0 max 16384 2024-05-17T04:41:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.998+0000 7eff85dd8740 1 Processor -- start 2024-05-17T04:41:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 -- start start 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e53f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e53d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff781ac7f0 con 0x55578e1a01b0 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff781aa060 con 0x55578e1772f0 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff85dd8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff781a7eb0 con 0x55578e17f4d0 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e53d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37328/0 (socket says 172.21.15.50:37328) 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e53d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51566/0 (socket says 172.21.15.50:51566) 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 -- 172.21.15.50:0/601560947 learned_addr learned my addr 172.21.15.50:0/601560947 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e53f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 -- 172.21.15.50:0/601560947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 msgr2=0x55578e53d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e53d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 -- 172.21.15.50:0/601560947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 msgr2=0x55578e53f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e53f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:23.999+0000 7eff7d3e3640 1 -- 172.21.15.50:0/601560947 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578e284620 con 0x55578e1a01b0 2024-05-17T04:41:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.000+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff700115e0 tx=0x7eff7001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5889fdec384de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.000+0000 7eff5e7fc640 1 -- 172.21.15.50:0/601560947 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7001acb0 con 0x55578e1a01b0 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.000+0000 7eff5e7fc640 1 -- 172.21.15.50:0/601560947 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff70024040 con 0x55578e1a01b0 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.000+0000 7eff5e7fc640 1 -- 172.21.15.50:0/601560947 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70004260 con 0x55578e1a01b0 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 -- 172.21.15.50:0/601560947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 msgr2=0x55578e53dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7eff700115e0 tx=0x7eff7001f040 comp rx=0 tx=0).stop 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 -- 172.21.15.50:0/601560947 shutdown_connections 2024-05-17T04:41:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e53dd20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e53d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 --2- 172.21.15.50:0/601560947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e53f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 -- 172.21.15.50:0/601560947 >> 172.21.15.50:0/601560947 conn(0x55578e194300 msgr2=0x55578e53eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.001+0000 7eff85dd8740 1 -- 172.21.15.50:0/601560947 shutdown_connections 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.002+0000 7eff85dd8740 1 -- 172.21.15.50:0/601560947 wait complete. 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.002+0000 7eff85dd8740 1 -- start start 2024-05-17T04:41:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.002+0000 7eff85dd8740 1 Processor -- start 2024-05-17T04:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e557ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e554a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e554fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e557ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff7d3e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e557ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:60684/0 (socket says 172.21.15.50:60684) 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff7d3e3640 1 -- 172.21.15.50:0/2304965517 learned_addr learned my addr 172.21.15.50:0/2304965517 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff781a8520 con 0x55578e1a01b0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff781a9990 con 0x55578e1772f0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff6ffff640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e554a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7eff781a8bf0 con 0x55578e17f4d0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.003+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e554fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 -- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 msgr2=0x55578e557ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e557ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff85dd8740 10 client.0 _ll_register_callbacks cb 0x7eff74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 -- 172.21.15.50:0/2304965517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 msgr2=0x55578e554fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e554fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578e5c4cd0 con 0x55578e17f4d0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff70024040 con 0x55578e17f4d0 2024-05-17T04:41:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff6ffff640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e554a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff68005760 tx=0x7eff680042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6615f56ebf1cc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.004+0000 7eff57fff640 10 client.25450 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.005+0000 7eff57fff640 1 -- 172.21.15.50:0/2304965517 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68035040 con 0x55578e17f4d0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.005+0000 7eff57fff640 1 -- 172.21.15.50:0/2304965517 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7eff680318c0 con 0x55578e17f4d0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.005+0000 7eff57fff640 1 -- 172.21.15.50:0/2304965517 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff68031b00 con 0x55578e17f4d0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.005+0000 7eff85dd8740 10 client.25450 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55578e5c5e50 con 0x55578e17f4d0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff57fff640 1 -- 172.21.15.50:0/2304965517 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7eff68031ca0 con 0x55578e17f4d0 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 10 client.25450 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 20 client.25450 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e1341c0 0x55578e5c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 4 client.25450 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.006+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55578e4b13f0 con 0x55578e1341c0 2024-05-17T04:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.007+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e1341c0 0x55578e5c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.007+0000 7eff57fff640 10 client.25450 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.007+0000 7eff7d3e3640 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e1341c0 0x55578e5c87b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a936fdfe0dfbcba server_cookie=2776036bcbddab59 in_seq=0 out_seq=0 2024-05-17T04:41:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff57fff640 1 -- 172.21.15.50:0/2304965517 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55578e4b13f0 con 0x55578e1341c0 2024-05-17T04:41:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff57fff640 10 client.25450 handle_command_reply: tid=0 2024-05-17T04:41:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 client.25450 shutdown 2024-05-17T04:41:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 msgr2=0x55578e554a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e554a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff68005760 tx=0x7eff680042d0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 shutdown_connections 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55578e1341c0 0x55578e5c87b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55578e1a01b0 0x55578e554fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578e17f4d0 0x55578e554a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 --2- 172.21.15.50:0/2304965517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55578e1772f0 0x55578e557ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff57fff640 0 client.25450 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 >> 172.21.15.50:0/2304965517 conn(0x55578e194300 msgr2=0x55578e541f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 shutdown_connections 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.031+0000 7eff85dd8740 1 -- 172.21.15.50:0/2304965517 wait complete. 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.032+0000 7eff85dd8740 20 client.25450 trim_cache size 0 max 16384 2024-05-17T04:41:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:41:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:41:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:41:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.032+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 msgr2=0x7eff600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.032+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 0x7eff600b4cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff740a6e40 tx=0x7eff7400ce60 comp rx=0 tx=0).stop 2024-05-17T04:41:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 msgr2=0x7eff78199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e9f0 tx=0x7eff7400eeb0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 shutdown_connections 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff78104c00 0x7eff781a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781042b0 0x7eff78199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7eff600b27e0 0x7eff600b4cf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 --2- 172.21.15.50:0/408465367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff78103190 0x7eff78199810 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 >> 172.21.15.50:0/408465367 conn(0x7eff780fe970 msgr2=0x7eff78107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 shutdown_connections 2024-05-17T04:41:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:24.033+0000 7eff7ee46640 1 -- 172.21.15.50:0/408465367 wait complete. 2024-05-17T04:41:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:24 smithi105 ceph-mon[34795]: pgmap v591: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-05-17T04:41:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:24 smithi110 ceph-mon[35497]: pgmap v591: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-05-17T04:41:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:24 smithi050 ceph-mon[31739]: pgmap v591: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.7 MiB/s wr, 366 op/s 2024-05-17T04:41:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:26 smithi105 ceph-mon[34795]: pgmap v592: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 363 op/s 2024-05-17T04:41:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:26 smithi110 ceph-mon[35497]: pgmap v592: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 363 op/s 2024-05-17T04:41:26.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:26 smithi050 ceph-mon[31739]: pgmap v592: 129 pgs: 129 active+clean; 959 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 363 op/s 2024-05-17T04:41:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 150.0 s, 151.4 tps, lat 675.818 ms stddev 280.369 2024-05-17T04:41:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:28 smithi105 ceph-mon[34795]: pgmap v593: 129 pgs: 129 active+clean; 960 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-17T04:41:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:28 smithi110 ceph-mon[35497]: pgmap v593: 129 pgs: 129 active+clean; 960 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-17T04:41:28.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:28 smithi050 ceph-mon[31739]: pgmap v593: 129 pgs: 129 active+clean; 960 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-17T04:41:30.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:30 smithi105 ceph-mon[34795]: pgmap v594: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.5 MiB/s wr, 363 op/s 2024-05-17T04:41:30.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:30 smithi110 ceph-mon[35497]: pgmap v594: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.5 MiB/s wr, 363 op/s 2024-05-17T04:41:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:30 smithi050 ceph-mon[31739]: pgmap v594: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.5 MiB/s wr, 363 op/s 2024-05-17T04:41:31.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:31 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:41:31.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:31 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:41:31.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:31 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:41:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: pgmap v595: 129 pgs: 129 active+clean; 976 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-17T04:41:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:41:32.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: pgmap v595: 129 pgs: 129 active+clean; 976 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:41:32.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: pgmap v595: 129 pgs: 129 active+clean; 976 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 388 op/s 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:41:32.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:41:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 155.0 s, 170.8 tps, lat 591.404 ms stddev 156.626 2024-05-17T04:41:33.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:33 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:41:33.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:33 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:41:33.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:33 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:41:33.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:33 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:41:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:33 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:41:33.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:33 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:41:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:34 smithi105 ceph-mon[34795]: pgmap v596: 129 pgs: 129 active+clean; 977 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-05-17T04:41:34.319 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:34 smithi110 ceph-mon[35497]: pgmap v596: 129 pgs: 129 active+clean; 977 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-05-17T04:41:34.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:34 smithi050 ceph-mon[31739]: pgmap v596: 129 pgs: 129 active+clean; 977 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.1 MiB/s wr, 351 op/s 2024-05-17T04:41:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:36 smithi105 ceph-mon[34795]: pgmap v597: 129 pgs: 129 active+clean; 981 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 378 op/s 2024-05-17T04:41:36.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:36 smithi110 ceph-mon[35497]: pgmap v597: 129 pgs: 129 active+clean; 981 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 378 op/s 2024-05-17T04:41:36.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:36 smithi050 ceph-mon[31739]: pgmap v597: 129 pgs: 129 active+clean; 981 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.0 MiB/s wr, 378 op/s 2024-05-17T04:41:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 -- 172.21.15.50:0/1450317528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 msgr2=0x7fe37c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 --2- 172.21.15.50:0/1450317528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c10b360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe374009be0 tx=0x7fe37402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 -- 172.21.15.50:0/1450317528 shutdown_connections 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 --2- 172.21.15.50:0/1450317528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c10b360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 --2- 172.21.15.50:0/1450317528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c1065e0 0x7fe37c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 --2- 172.21.15.50:0/1450317528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c103c80 0x7fe37c1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 -- 172.21.15.50:0/1450317528 >> 172.21.15.50:0/1450317528 conn(0x7fe37c0fd8b0 msgr2=0x7fe37c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 -- 172.21.15.50:0/1450317528 shutdown_connections 2024-05-17T04:41:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.160+0000 7fe3844e4640 1 -- 172.21.15.50:0/1450317528 wait complete. 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 Processor -- start 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 -- start start 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 0x7fe37c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c1065e0 0x7fe37c101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10eb60 con 0x7fe37c103c80 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10e9e0 con 0x7fe37c108f40 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.161+0000 7fe3844e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10ece0 con 0x7fe37c1065e0 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382280640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 0x7fe37c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:57942/0 (socket says 172.21.15.50:57942) 2024-05-17T04:41:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382280640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 0x7fe37c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39766/0 (socket says 172.21.15.50:39766) 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c1065e0 0x7fe37c101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 -- 172.21.15.50:0/934872675 learned_addr learned my addr 172.21.15.50:0/934872675 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 -- 172.21.15.50:0/934872675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c1065e0 msgr2=0x7fe37c101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c1065e0 0x7fe37c101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 -- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 msgr2=0x7fe37c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 0x7fe37c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 -- 172.21.15.50:0/934872675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c102800 con 0x7fe37c108f40 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.162+0000 7fe382a81640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe374030c10 tx=0x7fe374030c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.163+0000 7fe3737fe640 1 -- 172.21.15.50:0/934872675 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374041020 con 0x7fe37c108f40 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.163+0000 7fe3737fe640 1 -- 172.21.15.50:0/934872675 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe374037040 con 0x7fe37c108f40 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.163+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a3830 con 0x7fe37c108f40 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.163+0000 7fe3737fe640 1 -- 172.21.15.50:0/934872675 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374036070 con 0x7fe37c108f40 2024-05-17T04:41:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.163+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a3c90 con 0x7fe37c108f40 2024-05-17T04:41:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.166+0000 7fe38b3a3740 1 Processor -- start 2024-05-17T04:41:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.166+0000 7fe3737fe640 1 -- 172.21.15.50:0/934872675 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe374044020 con 0x7fe37c108f40 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 -- start start 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 0x564d651d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564d65144cc0 con 0x564d64e0e4d0 2024-05-17T04:41:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564d65134860 con 0x564d64e2f1b0 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d651388a0 con 0x564d64e062f0 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 0x564d651d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe382280640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 0x564d651d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:57944/0 (socket says 172.21.15.50:57944) 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe382280640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41810/0 (socket says 172.21.15.50:41810) 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.167+0000 7fe381a7f640 1 -- 172.21.15.50:0/2637438711 learned_addr learned my addr 172.21.15.50:0/2637438711 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe3737fe640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 0x7fe3680b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 -- 172.21.15.50:0/2637438711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 msgr2=0x564d651d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe381a7f640 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 0x564d651d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 -- 172.21.15.50:0/2637438711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 msgr2=0x564d651d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651d9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 -- 172.21.15.50:0/2637438711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d650fcd50 con 0x564d64e062f0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382a81640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 0x7fe3680b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe381a7f640 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe3737fe640 1 -- 172.21.15.50:0/934872675 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe374103060 con 0x7fe37c108f40 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382280640 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe37c06bd40 tx=0x7fe378011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5857b2ce69dabc68 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe370bfe640 1 -- 172.21.15.50:0/2637438711 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37801e070 con 0x564d64e062f0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe370bfe640 1 -- 172.21.15.50:0/2637438711 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3780198c0 con 0x564d64e062f0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe382a81640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 0x7fe3680b4d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ca280 tx=0x7fe374097f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.168+0000 7fe370bfe640 1 -- 172.21.15.50:0/2637438711 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37800e880 con 0x564d64e062f0 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.169+0000 7fe38b3a3740 1 -- 172.21.15.50:0/2637438711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 msgr2=0x564d651d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.169+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe37c06bd40 tx=0x7fe378011810 comp rx=0 tx=0).stop 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 -- 172.21.15.50:0/2637438711 shutdown_connections 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e062f0 0x564d651d2cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/2637438711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e2f1b0 0x564d651d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 -- 172.21.15.50:0/2637438711 >> 172.21.15.50:0/2637438711 conn(0x564d64e23300 msgr2=0x564d651ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 -- 172.21.15.50:0/2637438711 shutdown_connections 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 -- 172.21.15.50:0/2637438711 wait complete. 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 -- start start 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.170+0000 7fe38b3a3740 1 Processor -- start 2024-05-17T04:41:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 0x564d651dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564d65070ab0 con 0x564d64e062f0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564d650f14d0 con 0x564d64e0e4d0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564d651da690 con 0x564d64e2f1b0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe38b3a3740 10 client.0 _ll_register_callbacks cb 0x564d6512f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39794/0 (socket says 172.21.15.50:39794) 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 -- 172.21.15.50:0/942218986 learned_addr learned my addr 172.21.15.50:0/942218986 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe382280640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 0x564d651dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 -- 172.21.15.50:0/942218986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 msgr2=0x564d651e3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 -- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 msgr2=0x564d651dc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 0x564d651dc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 -- 172.21.15.50:0/942218986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d650f2340 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe382280640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 0x564d651dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe381a7f640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009560 tx=0x7fe36c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8a4faabcc0eced server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.171+0000 7fe365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00fe60 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe36c004030 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3780198c0 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe36c027050 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe38b3a3740 10 client.15708 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d64e9a790 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3780198c0 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.172+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe36c003b10 con 0x564d64e062f0 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 10 client.15708 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 20 client.15708 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dedd80 0x564d651e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 4 client.15708 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d65146c10 con 0x564d64dedd80 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe382280640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dedd80 0x564d651e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe382280640 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dedd80 0x564d651e9890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45042a1b14c14a32 server_cookie=3db66f8ab1d121c8 in_seq=0 out_seq=0 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.173+0000 7fe365ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe365ffb640 1 -- 172.21.15.50:0/942218986 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d65146c10 con 0x564d64dedd80 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe365ffb640 10 client.15708 handle_command_reply: tid=0 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 client.15708 shutdown 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 msgr2=0x564d651dc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009560 tx=0x7fe36c0098d0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 shutdown_connections 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e0e4d0 0x564d651dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dedd80 0x564d651e9890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/942218986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e062f0 0x564d651dc060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 >> 172.21.15.50:0/942218986 conn(0x564d64e23300 msgr2=0x564d651d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.174+0000 7fe365ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.175+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 shutdown_connections 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.175+0000 7fe38b3a3740 1 -- 172.21.15.50:0/942218986 wait complete. 2024-05-17T04:41:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.175+0000 7fe38b3a3740 20 client.15708 trim_cache size 0 max 16384 2024-05-17T04:41:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 Processor -- start 2024-05-17T04:41:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 -- start start 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b0a20 con 0x564d64e0e4d0 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ae290 con 0x564d64e062f0 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ac0e0 con 0x564d64e2f1b0 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe381a7f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:57968/0 (socket says 172.21.15.50:57968) 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe381a7f640 1 -- 172.21.15.50:0/4211592466 learned_addr learned my addr 172.21.15.50:0/4211592466 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe382280640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.196+0000 7fe381a7f640 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 -- 172.21.15.50:0/4211592466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 msgr2=0x564d651e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 -- 172.21.15.50:0/4211592466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 msgr2=0x564d651e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 -- 172.21.15.50:0/4211592466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d64f13620 con 0x564d64e062f0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe382280640 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe381a7f640 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe36c010b50 tx=0x7fe36c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e927d6524833aaff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe367fff640 1 -- 172.21.15.50:0/4211592466 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c00fe60 con 0x564d64e062f0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe367fff640 1 -- 172.21.15.50:0/4211592466 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36c004490 con 0x564d64e062f0 2024-05-17T04:41:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.197+0000 7fe367fff640 1 -- 172.21.15.50:0/4211592466 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c005290 con 0x564d64e062f0 2024-05-17T04:41:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 -- 172.21.15.50:0/4211592466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 msgr2=0x564d651ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe36c010b50 tx=0x7fe36c001e70 comp rx=0 tx=0).stop 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 -- 172.21.15.50:0/4211592466 shutdown_connections 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e2f1b0 0x564d651e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e0e4d0 0x564d651e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/4211592466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651ce270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 -- 172.21.15.50:0/4211592466 >> 172.21.15.50:0/4211592466 conn(0x564d64e23300 msgr2=0x564d651cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.198+0000 7fe38b3a3740 1 -- 172.21.15.50:0/4211592466 shutdown_connections 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.199+0000 7fe38b3a3740 1 -- 172.21.15.50:0/4211592466 wait complete. 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.199+0000 7fe38b3a3740 1 -- start start 2024-05-17T04:41:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.199+0000 7fe38b3a3740 1 Processor -- start 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e2f1b0 0x564d651e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ac750 con 0x564d64e2f1b0 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1adbc0 con 0x564d64e062f0 2024-05-17T04:41:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe38b3a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ace20 con 0x564d64e0e4d0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe382280640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe382280640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41838/0 (socket says 172.21.15.50:41838) 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.200+0000 7fe382280640 1 -- 172.21.15.50:0/437406427 learned_addr learned my addr 172.21.15.50:0/437406427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe381a7f640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e2f1b0 0x564d651e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe381a7f640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe38b3a3740 10 client.0 _ll_register_callbacks cb 0x7fe36c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 -- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 msgr2=0x564d651e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 -- 172.21.15.50:0/437406427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e2f1b0 msgr2=0x564d651e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e2f1b0 0x564d651e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 -- 172.21.15.50:0/437406427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d6524fc30 con 0x564d64e0e4d0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe382280640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe378005fd0 tx=0x7fe378009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2a5868837cc0f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37801e040 con 0x564d64e0e4d0 2024-05-17T04:41:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c004490 con 0x564d64e0e4d0 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.201+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe378005040 con 0x564d64e0e4d0 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.202+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe37803c050 con 0x564d64e0e4d0 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.202+0000 7fe38b3a3740 10 client.25468 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.202+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d65250e50 con 0x564d64e0e4d0 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.202+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36c004490 con 0x564d64e0e4d0 2024-05-17T04:41:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe37802d640 con 0x564d64e0e4d0 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 10 client.25468 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 20 client.25468 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dc31c0 0x564d65253730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 4 client.25468 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d651403f0 con 0x564d64dc31c0 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.203+0000 7fe381a7f640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dc31c0 0x564d65253730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.204+0000 7fe365ffb640 10 client.25468 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.204+0000 7fe381a7f640 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dc31c0 0x564d65253730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5aa0cc487c84eb server_cookie=ab0e9ca56c64b4fe in_seq=0 out_seq=0 2024-05-17T04:41:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe365ffb640 1 -- 172.21.15.50:0/437406427 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564d651403f0 con 0x564d64dc31c0 2024-05-17T04:41:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe365ffb640 10 client.25468 handle_command_reply: tid=0 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 client.25468 shutdown 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 msgr2=0x564d651e0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe378005fd0 tx=0x7fe378009fb0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 shutdown_connections 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564d64dc31c0 0x564d65253730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d64e2f1b0 0x564d651e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d64e0e4d0 0x564d651e0780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 --2- 172.21.15.50:0/437406427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d64e062f0 0x564d651e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 >> 172.21.15.50:0/437406427 conn(0x564d64e23300 msgr2=0x564d651d5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe365ffb640 0 client.25468 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 shutdown_connections 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 1 -- 172.21.15.50:0/437406427 wait complete. 2024-05-17T04:41:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.219+0000 7fe38b3a3740 20 client.25468 trim_cache size 0 max 16384 2024-05-17T04:41:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:41:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:41:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 msgr2=0x7fe3680b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 0x7fe3680b4d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ca280 tx=0x7fe374097f10 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 msgr2=0x7fe37c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe374030c10 tx=0x7fe374030c40 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 shutdown_connections 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe37c108f40 0x7fe37c1020d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe37c1065e0 0x7fe37c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3680b2860 0x7fe3680b4d70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 --2- 172.21.15.50:0/934872675 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe37c103c80 0x7fe37c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 >> 172.21.15.50:0/934872675 conn(0x7fe37c0fd8b0 msgr2=0x7fe37c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 shutdown_connections 2024-05-17T04:41:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:37.220+0000 7fe3844e4640 1 -- 172.21.15.50:0/934872675 wait complete. 2024-05-17T04:41:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 160.0 s, 146.6 tps, lat 670.135 ms stddev 254.539 2024-05-17T04:41:38.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:38 smithi105 ceph-mon[34795]: pgmap v598: 129 pgs: 129 active+clean; 985 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 346 op/s 2024-05-17T04:41:38.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:38 smithi110 ceph-mon[35497]: pgmap v598: 129 pgs: 129 active+clean; 985 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 346 op/s 2024-05-17T04:41:38.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:38 smithi050 ceph-mon[31739]: pgmap v598: 129 pgs: 129 active+clean; 985 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 346 op/s 2024-05-17T04:41:40.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:40 smithi105 ceph-mon[34795]: pgmap v599: 129 pgs: 129 active+clean; 993 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 385 op/s 2024-05-17T04:41:40.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:40 smithi110 ceph-mon[35497]: pgmap v599: 129 pgs: 129 active+clean; 993 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 385 op/s 2024-05-17T04:41:40.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:40 smithi050 ceph-mon[31739]: pgmap v599: 129 pgs: 129 active+clean; 993 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 385 op/s 2024-05-17T04:41:42.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:42 smithi105 ceph-mon[34795]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.7 MiB/s wr, 365 op/s 2024-05-17T04:41:42.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:42 smithi110 ceph-mon[35497]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.7 MiB/s wr, 365 op/s 2024-05-17T04:41:42.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:42 smithi050 ceph-mon[31739]: pgmap v600: 129 pgs: 129 active+clean; 993 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.7 MiB/s wr, 365 op/s 2024-05-17T04:41:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 165.0 s, 145.2 tps, lat 683.234 ms stddev 178.813 2024-05-17T04:41:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:44 smithi105 ceph-mon[34795]: pgmap v601: 129 pgs: 129 active+clean; 994 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-05-17T04:41:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:44 smithi110 ceph-mon[35497]: pgmap v601: 129 pgs: 129 active+clean; 994 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-05-17T04:41:44.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:44 smithi050 ceph-mon[31739]: pgmap v601: 129 pgs: 129 active+clean; 994 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.4 MiB/s wr, 394 op/s 2024-05-17T04:41:46.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:46 smithi105 ceph-mon[34795]: pgmap v602: 129 pgs: 129 active+clean; 1002 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 375 op/s 2024-05-17T04:41:46.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:46 smithi110 ceph-mon[35497]: pgmap v602: 129 pgs: 129 active+clean; 1002 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 375 op/s 2024-05-17T04:41:46.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:46 smithi050 ceph-mon[31739]: pgmap v602: 129 pgs: 129 active+clean; 1002 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 375 op/s 2024-05-17T04:41:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 170.0 s, 135.2 tps, lat 729.953 ms stddev 218.380 2024-05-17T04:41:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:48 smithi105 ceph-mon[34795]: pgmap v603: 129 pgs: 129 active+clean; 1006 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.2 MiB/s wr, 356 op/s 2024-05-17T04:41:48.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:48 smithi110 ceph-mon[35497]: pgmap v603: 129 pgs: 129 active+clean; 1006 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.2 MiB/s wr, 356 op/s 2024-05-17T04:41:48.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:48 smithi050 ceph-mon[31739]: pgmap v603: 129 pgs: 129 active+clean; 1006 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.2 MiB/s wr, 356 op/s 2024-05-17T04:41:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:50 smithi105 ceph-mon[34795]: pgmap v604: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.2 MiB/s wr, 378 op/s 2024-05-17T04:41:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:50 smithi110 ceph-mon[35497]: pgmap v604: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.2 MiB/s wr, 378 op/s 2024-05-17T04:41:50.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:50 smithi050 ceph-mon[31739]: pgmap v604: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.2 MiB/s wr, 378 op/s 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 -- 172.21.15.50:0/1879155243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c101a60 msgr2=0x7f549c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 --2- 172.21.15.50:0/1879155243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c101a60 0x7f549c101e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f548c009900 tx=0x7f548c02a410 comp rx=0 tx=0).stop 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 -- 172.21.15.50:0/1879155243 shutdown_connections 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 --2- 172.21.15.50:0/1879155243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c102d00 0x7f549c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 --2- 172.21.15.50:0/1879155243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c1023b0 0x7f549c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 --2- 172.21.15.50:0/1879155243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c101a60 0x7f549c101e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 -- 172.21.15.50:0/1879155243 >> 172.21.15.50:0/1879155243 conn(0x7f549c0fd880 msgr2=0x7f549c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 -- 172.21.15.50:0/1879155243 shutdown_connections 2024-05-17T04:41:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.359+0000 7f54a1047640 1 -- 172.21.15.50:0/1879155243 wait complete. 2024-05-17T04:41:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 Processor -- start 2024-05-17T04:41:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 -- start start 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c101a60 0x7f549c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 0x7f549c19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 0x7f549c1a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f549c110d50 con 0x7f549c102d00 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c110bd0 con 0x7f549c1023b0 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f54a1047640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f549c110ed0 con 0x7f549c101a60 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f549b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 0x7f549c1a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c101a60 0x7f549c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f549a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 0x7f549c19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f549b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 0x7f549c1a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41860/0 (socket says 172.21.15.50:41860) 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.360+0000 7f549b59e640 1 -- 172.21.15.50:0/3763106209 learned_addr learned my addr 172.21.15.50:0/3763106209 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 -- 172.21.15.50:0/3763106209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c101a60 msgr2=0x7f549c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c101a60 0x7f549c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 -- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 msgr2=0x7f549c1a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 0x7f549c1a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 -- 172.21.15.50:0/3763106209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c1a3570 con 0x7f549c1023b0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f549a59c640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 0x7f549c19bf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f5483fff640 1 -- 172.21.15.50:0/3763106209 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000ccc0 con 0x7f549c1023b0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c1a3860 con 0x7f549c1023b0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f5483fff640 1 -- 172.21.15.50:0/3763106209 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5490005020 con 0x7f549c1023b0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f5483fff640 1 -- 172.21.15.50:0/3763106209 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000ce60 con 0x7f549c1023b0 2024-05-17T04:41:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.361+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a3db0 con 0x7f549c1023b0 2024-05-17T04:41:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 Processor -- start 2024-05-17T04:41:50.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 -- start start 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f5483fff640 1 -- 172.21.15.50:0/3763106209 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5490004a40 con 0x7f549c1023b0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f85770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560e40ef7cc0 con 0x560e40bb92f0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560e40ee7860 con 0x560e40bc14d0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.364+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e40eeb8a0 con 0x560e40be21b0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f85770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f85770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:32868/0 (socket says 172.21.15.50:32868) 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549b59e640 1 -- 172.21.15.50:0/676389951 learned_addr learned my addr 172.21.15.50:0/676389951 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549a59c640 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 -- 172.21.15.50:0/676389951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 msgr2=0x560e40f85770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f85770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 -- 172.21.15.50:0/676389951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 msgr2=0x560e40f8cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 -- 172.21.15.50:0/676389951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e40eafd50 con 0x560e40bb92f0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f5483fff640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 0x7f54840b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549ad9d640 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f85cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f548c009230 tx=0x7f548c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11798db940a937b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f549b59e640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 0x7f54840b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.365+0000 7f5483fff640 1 -- 172.21.15.50:0/3763106209 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f54900e2540 con 0x7f549c1023b0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.366+0000 7f54817fa640 1 -- 172.21.15.50:0/676389951 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c035030 con 0x560e40bb92f0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.366+0000 7f54817fa640 1 -- 172.21.15.50:0/676389951 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f548c040070 con 0x560e40bb92f0 2024-05-17T04:41:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.366+0000 7f54817fa640 1 -- 172.21.15.50:0/676389951 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c003850 con 0x560e40bb92f0 2024-05-17T04:41:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f549b59e640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 0x7f54840b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560e40f86960 tx=0x7f5494009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 -- 172.21.15.50:0/676389951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 msgr2=0x560e40f85cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f85cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f548c009230 tx=0x7f548c005fd0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 -- 172.21.15.50:0/676389951 shutdown_connections 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f85cf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/676389951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f85770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 -- 172.21.15.50:0/676389951 >> 172.21.15.50:0/676389951 conn(0x560e40bd6300 msgr2=0x560e40f811a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 -- 172.21.15.50:0/676389951 shutdown_connections 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.367+0000 7f54a7fe5740 1 -- 172.21.15.50:0/676389951 wait complete. 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 -- start start 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 Processor -- start 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40bb92f0 0x560e40f8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560e40e23ab0 con 0x560e40be21b0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560e40ea44d0 con 0x560e40bc14d0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e40f8d630 con 0x560e40bb92f0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f549b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f549b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41872/0 (socket says 172.21.15.50:41872) 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.368+0000 7f549b59e640 1 -- 172.21.15.50:0/2885417220 learned_addr learned my addr 172.21.15.50:0/2885417220 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f54a7fe5740 10 client.0 _ll_register_callbacks cb 0x560e40ee2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40bb92f0 0x560e40f8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549a59c640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 -- 172.21.15.50:0/2885417220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40bb92f0 msgr2=0x560e40f8efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40bb92f0 0x560e40f8efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 -- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 msgr2=0x560e40f8f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 -- 172.21.15.50:0/2885417220 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e40ea5340 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f549b59e640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f549400f840 tx=0x7f5494004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cdbfdfc2aad5c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f547eff5640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.369+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940119c0 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5494011b60 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c040070 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5494031050 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 10 client.15726 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560e40c4d790 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f548c040070 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f54940044b0 con 0x560e40be21b0 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 10 client.15726 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 20 client.15726 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.370+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40ba0d80 0x560e40f9c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.371+0000 7f54a7fe5740 4 client.15726 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:41:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.371+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e40ef9c10 con 0x560e40ba0d80 2024-05-17T04:41:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.371+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40ba0d80 0x560e40f9c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.371+0000 7f547eff5640 10 client.15726 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.371+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40ba0d80 0x560e40f9c920 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d6a3deb9e7dc29 server_cookie=ff95d0cb0b2a55a1 in_seq=0 out_seq=0 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.372+0000 7f547eff5640 1 -- 172.21.15.50:0/2885417220 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e40ef9c10 con 0x560e40ba0d80 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.372+0000 7f547eff5640 10 client.15726 handle_command_reply: tid=0 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.372+0000 7f54a7fe5740 1 client.15726 shutdown 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.372+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 msgr2=0x560e40f96470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.372+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f549400f840 tx=0x7f5494004ab0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 shutdown_connections 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40ba0d80 0x560e40f9c920 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40be21b0 0x560e40f96470 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f8f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2885417220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40bb92f0 0x560e40f8efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 >> 172.21.15.50:0/2885417220 conn(0x560e40bd6300 msgr2=0x560e40f84010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f547eff5640 0 client.15726 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 shutdown_connections 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2885417220 wait complete. 2024-05-17T04:41:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.373+0000 7f54a7fe5740 20 client.15726 trim_cache size 0 max 16384 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.395+0000 7f54a7fe5740 1 Processor -- start 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.395+0000 7f54a7fe5740 1 -- start start 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bc14d0 0x560e40f96400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f96940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1b0ab0 con 0x560e40bc14d0 2024-05-17T04:41:50.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ae320 con 0x560e40bb92f0 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ac170 con 0x560e40be21b0 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f96940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:55366/0 (socket says 172.21.15.50:55366) 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 -- 172.21.15.50:0/4023393359 learned_addr learned my addr 172.21.15.50:0/4023393359 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549a59c640 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bc14d0 0x560e40f96400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 -- 172.21.15.50:0/4023393359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 msgr2=0x560e40f96940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f96940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 -- 172.21.15.50:0/4023393359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bc14d0 msgr2=0x560e40f96400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bc14d0 0x560e40f96400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549ad9d640 1 -- 172.21.15.50:0/4023393359 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e40cc6620 con 0x560e40bb92f0 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.396+0000 7f549b59e640 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f96940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:41:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.397+0000 7f549ad9d640 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f548c009230 tx=0x7f548c031420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbf49ce9c12c09a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.397+0000 7f5480ff9640 1 -- 172.21.15.50:0/4023393359 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c055020 con 0x560e40bb92f0 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.397+0000 7f5480ff9640 1 -- 172.21.15.50:0/4023393359 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f548c040040 con 0x560e40bb92f0 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.397+0000 7f5480ff9640 1 -- 172.21.15.50:0/4023393359 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c035040 con 0x560e40bb92f0 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- 172.21.15.50:0/4023393359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 msgr2=0x560e40f81140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f548c009230 tx=0x7f548c031420 comp rx=0 tx=0).stop 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- 172.21.15.50:0/4023393359 shutdown_connections 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f96940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bc14d0 0x560e40f96400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/4023393359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bb92f0 0x560e40f81140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- 172.21.15.50:0/4023393359 >> 172.21.15.50:0/4023393359 conn(0x560e40bd6300 msgr2=0x560e40f80a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- 172.21.15.50:0/4023393359 shutdown_connections 2024-05-17T04:41:50.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- 172.21.15.50:0/4023393359 wait complete. 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 -- start start 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.398+0000 7f54a7fe5740 1 Processor -- start 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f93930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ac7e0 con 0x560e40bb92f0 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1adc50 con 0x560e40bc14d0 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1aceb0 con 0x560e40be21b0 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41890/0 (socket says 172.21.15.50:41890) 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 -- 172.21.15.50:0/2791104710 learned_addr learned my addr 172.21.15.50:0/2791104710 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549a59c640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549b59e640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f93930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f54a7fe5740 10 client.0 _ll_register_callbacks cb 0x7f5494012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 -- 172.21.15.50:0/2791104710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 msgr2=0x560e40f93930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f93930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 -- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 msgr2=0x560e40f933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.399+0000 7f549ad9d640 1 -- 172.21.15.50:0/2791104710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e40f96ea0 con 0x560e40bb92f0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f549ad9d640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f548c02a930 tx=0x7f548c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89491386941fd0be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f547eff5640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c060060 con 0x560e40bb92f0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f548c031c70 con 0x560e40bb92f0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c040040 con 0x560e40bb92f0 2024-05-17T04:41:50.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f548c035040 con 0x560e40bb92f0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f54a7fe5740 10 client.15732 fetch_fsmap learned FSMap version 37 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560e41003b50 con 0x560e40bb92f0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.400+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f548c040040 con 0x560e40bb92f0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f548c05a070 con 0x560e40bb92f0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 10 client.15732 fetch_fsmap finished waiting for FSMap version 37 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 20 client.15732 populate_metadata read hostname 'smithi050' 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40b761c0 0x560e410060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 4 client.15732 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e40ef33f0 con 0x560e40b761c0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f549a59c640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40b761c0 0x560e410060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:41:50.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.401+0000 7f547eff5640 10 client.15732 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.402+0000 7f549a59c640 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40b761c0 0x560e410060b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c53bef1caebc922 server_cookie=74d08b16fb76b9b7 in_seq=0 out_seq=0 2024-05-17T04:41:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.415+0000 7f547eff5640 1 -- 172.21.15.50:0/2791104710 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560e40ef33f0 con 0x560e40b761c0 2024-05-17T04:41:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.415+0000 7f547eff5640 10 client.15732 handle_command_reply: tid=0 2024-05-17T04:41:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.415+0000 7f54a7fe5740 1 client.15732 shutdown 2024-05-17T04:41:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 msgr2=0x560e40f96800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f548c02a930 tx=0x7f548c000bc0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 shutdown_connections 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e40be21b0 0x560e40f93930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e40bc14d0 0x560e40f933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560e40b761c0 0x560e410060b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 --2- 172.21.15.50:0/2791104710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560e40bb92f0 0x560e40f96800 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f547eff5640 0 client.15732 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 >> 172.21.15.50:0/2791104710 conn(0x560e40bd6300 msgr2=0x560e40f88bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 shutdown_connections 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 1 -- 172.21.15.50:0/2791104710 wait complete. 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.416+0000 7f54a7fe5740 20 client.15732 trim_cache size 0 max 16384 2024-05-17T04:41:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:41:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:41:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.417+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 msgr2=0x7f54840b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.417+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 0x7f54840b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560e40f86960 tx=0x7f5494009500 comp rx=0 tx=0).stop 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.417+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 msgr2=0x7f549c19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.417+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 0x7f549c19bf70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f549000e9f0 tx=0x7f549000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 shutdown_connections 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f54840b27e0 0x7f54840b4cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f549c102d00 0x7f549c1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f549c1023b0 0x7f549c19bf70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 --2- 172.21.15.50:0/3763106209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f549c101a60 0x7f549c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:41:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 >> 172.21.15.50:0/3763106209 conn(0x7f549c0fd880 msgr2=0x7f549c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:41:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 shutdown_connections 2024-05-17T04:41:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:41:50.418+0000 7f54a1047640 1 -- 172.21.15.50:0/3763106209 wait complete. 2024-05-17T04:41:52.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:52 smithi105 ceph-mon[34795]: pgmap v605: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-17T04:41:52.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:52 smithi110 ceph-mon[35497]: pgmap v605: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-17T04:41:52.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:52 smithi050 ceph-mon[31739]: pgmap v605: 129 pgs: 129 active+clean; 1010 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.8 MiB/s wr, 316 op/s 2024-05-17T04:41:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 175.0 s, 149.0 tps, lat 687.994 ms stddev 216.537 2024-05-17T04:41:54.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:54 smithi110 ceph-mon[35497]: pgmap v606: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-05-17T04:41:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:54 smithi050 ceph-mon[31739]: pgmap v606: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-05-17T04:41:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:54 smithi105 ceph-mon[34795]: pgmap v606: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 365 op/s 2024-05-17T04:41:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:56 smithi050 ceph-mon[31739]: pgmap v607: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-17T04:41:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:56 smithi105 ceph-mon[34795]: pgmap v607: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-17T04:41:56.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:56 smithi110 ceph-mon[35497]: pgmap v607: 129 pgs: 129 active+clean; 1011 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 338 op/s 2024-05-17T04:41:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 180.0 s, 141.8 tps, lat 700.207 ms stddev 225.394 2024-05-17T04:41:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:41:58 smithi050 ceph-mon[31739]: pgmap v608: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.6 MiB/s wr, 305 op/s 2024-05-17T04:41:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:41:58 smithi105 ceph-mon[34795]: pgmap v608: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.6 MiB/s wr, 305 op/s 2024-05-17T04:41:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:41:58 smithi110 ceph-mon[35497]: pgmap v608: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.6 MiB/s wr, 305 op/s 2024-05-17T04:42:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:00 smithi050 ceph-mon[31739]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:00 smithi105 ceph-mon[34795]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:00 smithi110 ceph-mon[35497]: pgmap v609: 129 pgs: 129 active+clean; 1.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:02 smithi050 ceph-mon[31739]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-17T04:42:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:02 smithi105 ceph-mon[34795]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-17T04:42:02.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:02 smithi110 ceph-mon[35497]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.4 MiB/s wr, 302 op/s 2024-05-17T04:42:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 185.0 s, 146.0 tps, lat 691.949 ms stddev 160.328 2024-05-17T04:42:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 -- 172.21.15.50:0/23116189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d0102080 msgr2=0x7f97d0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 --2- 172.21.15.50:0/23116189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d0102080 0x7f97d0102490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009900 tx=0x7f97c002a3f0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 -- 172.21.15.50:0/23116189 shutdown_connections 2024-05-17T04:42:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 --2- 172.21.15.50:0/23116189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 0x7f97d010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 --2- 172.21.15.50:0/23116189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d01031a0 0x7f97d01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 --2- 172.21.15.50:0/23116189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d0102080 0x7f97d0102490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 -- 172.21.15.50:0/23116189 >> 172.21.15.50:0/23116189 conn(0x7f97d00fd880 msgr2=0x7f97d00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 -- 172.21.15.50:0/23116189 shutdown_connections 2024-05-17T04:42:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.549+0000 7f97d55cb640 1 -- 172.21.15.50:0/23116189 wait complete. 2024-05-17T04:42:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 Processor -- start 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 -- start start 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d0102080 0x7f97d01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 0x7f97d01a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97d010daf0 con 0x7f97d0102080 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97d010d970 con 0x7f97d01031a0 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.550+0000 7f97d55cb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97d010dc70 con 0x7f97d0103af0 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d0102080 0x7f97d01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44496/0 (socket says 172.21.15.50:44496) 2024-05-17T04:42:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 -- 172.21.15.50:0/3198014156 learned_addr learned my addr 172.21.15.50:0/3198014156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 0x7f97d01a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 -- 172.21.15.50:0/3198014156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 msgr2=0x7f97d01a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 0x7f97d01a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 -- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d0102080 msgr2=0x7f97d01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d0102080 0x7f97d01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.551+0000 7f97ce7fc640 1 -- 172.21.15.50:0/3198014156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d01a1410 con 0x7f97d01031a0 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97c400b4a0 tx=0x7f97c400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97b7fff640 1 -- 172.21.15.50:0/3198014156 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4014070 con 0x7f97d01031a0 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97b7fff640 1 -- 172.21.15.50:0/3198014156 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c4010aa0 con 0x7f97d01031a0 2024-05-17T04:42:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d01a1700 con 0x7f97d01031a0 2024-05-17T04:42:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97b7fff640 1 -- 172.21.15.50:0/3198014156 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4017b70 con 0x7f97d01031a0 2024-05-17T04:42:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.552+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01a1d20 con 0x7f97d01031a0 2024-05-17T04:42:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.556+0000 7f97b7fff640 1 -- 172.21.15.50:0/3198014156 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f97c4017070 con 0x7f97d01031a0 2024-05-17T04:42:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.556+0000 7f97dc122740 1 Processor -- start 2024-05-17T04:42:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.556+0000 7f97dc122740 1 -- start start 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946efa0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37352/0 (socket says 172.21.15.50:37352) 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97cf7fe640 1 -- 172.21.15.50:0/1085766394 learned_addr learned my addr 172.21.15.50:0/1085766394 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97ceffd640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946efa0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efa7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55946ef12cc0 con 0x55946ebd42f0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55946ef02860 con 0x55946ebdc4d0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55946ef068a0 con 0x55946ebfd1b0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.557+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efa7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 -- 172.21.15.50:0/1085766394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 msgr2=0x55946efa0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 -- 172.21.15.50:0/1085766394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 msgr2=0x55946efa7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efa7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 -- 172.21.15.50:0/1085766394 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946eecad50 con 0x55946ebd42f0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97b7fff640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 0x7f97b80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 0x7f97b80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97b7fff640 1 -- 172.21.15.50:0/3198014156 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97c40e1c60 con 0x7f97d01031a0 2024-05-17T04:42:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97ceffd640 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946efa0cf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009230 tx=0x7f97c0004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0f15454a3f7678 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.558+0000 7f97b57fa640 1 -- 172.21.15.50:0/1085766394 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0035030 con 0x55946ebd42f0 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.559+0000 7f97b57fa640 1 -- 172.21.15.50:0/1085766394 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c0040070 con 0x55946ebd42f0 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.559+0000 7f97b57fa640 1 -- 172.21.15.50:0/1085766394 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0031a50 con 0x55946ebd42f0 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 msgr2=0x55946efa0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 0x7f97b80b4d70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55946efa1960 tx=0x7f97c80097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946efa0cf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009230 tx=0x7f97c0004330 comp rx=0 tx=0).stop 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 shutdown_connections 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efa7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946efa0cf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 --2- 172.21.15.50:0/1085766394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efa0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 >> 172.21.15.50:0/1085766394 conn(0x55946ebf1300 msgr2=0x55946ef9c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 shutdown_connections 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- 172.21.15.50:0/1085766394 wait complete. 2024-05-17T04:42:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.560+0000 7f97dc122740 1 -- start start 2024-05-17T04:42:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.561+0000 7f97dc122740 1 Processor -- start 2024-05-17T04:42:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.561+0000 7f97dc122740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 0x55946efaa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.561+0000 7f97dc122740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebdc4d0 0x55946efaa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97dc122740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97dc122740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55946ee3eab0 con 0x55946ebfd1b0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39990/0 (socket says 172.21.15.50:39990) 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97cf7fe640 1 -- 172.21.15.50:0/3029790567 learned_addr learned my addr 172.21.15.50:0/3029790567 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebdc4d0 0x55946efaa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97ceffd640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 0x55946efaa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55946eebf4d0 con 0x55946ebd42f0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.562+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55946efa8630 con 0x55946ebdc4d0 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 -- 172.21.15.50:0/3029790567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebdc4d0 msgr2=0x55946efaa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebdc4d0 0x55946efaa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 -- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 msgr2=0x55946efaa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 0x55946efaa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946eec0340 con 0x55946ebfd1b0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97ceffd640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 0x55946efaa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f97c8005fd0 tx=0x7f97c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2356dba1a9b4de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97dc122740 10 client.0 _ll_register_callbacks cb 0x55946eefd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c0040070 con 0x55946ebfd1b0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97b2ff5640 1 -- 172.21.15.50:0/3029790567 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c800ecf0 con 0x55946ebfd1b0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.563+0000 7f97b2ff5640 1 -- 172.21.15.50:0/3029790567 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97c801f420 con 0x55946ebfd1b0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97b2ff5640 1 -- 172.21.15.50:0/3029790567 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f97c8030050 con 0x55946ebfd1b0 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 10 client.15741 fetch_fsmap learned FSMap version 38 2024-05-17T04:42:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55946ec68790 con 0x55946ebfd1b0 2024-05-17T04:42:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97b2ff5640 1 -- 172.21.15.50:0/3029790567 <== mon.0 v2:172.21.15.50:3300/0 4 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f97c800f040 con 0x55946ebfd1b0 2024-05-17T04:42:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 10 client.15741 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T04:42:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 20 client.15741 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946ebbbd80 0x55946efb7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 4 client.15741 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.564+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946ef14c10 con 0x55946ebbbd80 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.565+0000 7f97ceffd640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946ebbbd80 0x55946efb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.566+0000 7f97b2ff5640 10 client.15741 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.566+0000 7f97ceffd640 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946ebbbd80 0x55946efb7600 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fdee655064d1a08 server_cookie=f9b992cea5c82f21 in_seq=0 out_seq=0 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97b2ff5640 1 -- 172.21.15.50:0/3029790567 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55946ef14c10 con 0x55946ebbbd80 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97b2ff5640 10 client.15741 handle_command_reply: tid=0 2024-05-17T04:42:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 client.15741 shutdown 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 msgr2=0x55946efb14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f97c8005fd0 tx=0x7f97c8004a40 comp rx=0 tx=0).stop 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 shutdown_connections 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946ebbbd80 0x55946efb7600 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efb14a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebdc4d0 0x55946efaa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 --2- 172.21.15.50:0/3029790567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebd42f0 0x55946efaa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97b2ff5640 0 client.15741 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.567+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 >> 172.21.15.50:0/3029790567 conn(0x55946ebf1300 msgr2=0x55946ef9f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.568+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 shutdown_connections 2024-05-17T04:42:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.568+0000 7f97dc122740 1 -- 172.21.15.50:0/3029790567 wait complete. 2024-05-17T04:42:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.568+0000 7f97dc122740 20 client.15741 trim_cache size 0 max 16384 2024-05-17T04:42:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.590+0000 7f97dc122740 1 Processor -- start 2024-05-17T04:42:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.590+0000 7f97dc122740 1 -- start start 2024-05-17T04:42:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.590+0000 7f97dc122740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.590+0000 7f97dc122740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946ef9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97dc122740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efb1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97dc122740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01ae9b0 con 0x55946ebd42f0 2024-05-17T04:42:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97dc122740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01ac250 con 0x55946ebdc4d0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97dc122740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01aa0a0 con 0x55946ebfd1b0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40004/0 (socket says 172.21.15.50:40004) 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 -- 172.21.15.50:0/2111047081 learned_addr learned my addr 172.21.15.50:0/2111047081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efb1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946ef9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 -- 172.21.15.50:0/2111047081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 msgr2=0x55946efb1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efb1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 -- 172.21.15.50:0/2111047081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 msgr2=0x55946ef9fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946ef9fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ceffd640 1 -- 172.21.15.50:0/2111047081 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946ece1620 con 0x55946ebd42f0 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.591+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946ef9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.592+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97c002a9e0 tx=0x7f97c0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afe20d6184507e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.592+0000 7f97b4ff9640 1 -- 172.21.15.50:0/2111047081 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0035050 con 0x55946ebd42f0 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.592+0000 7f97b4ff9640 1 -- 172.21.15.50:0/2111047081 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c0053070 con 0x55946ebd42f0 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.592+0000 7f97b4ff9640 1 -- 172.21.15.50:0/2111047081 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0045470 con 0x55946ebd42f0 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.593+0000 7f97dc122740 1 -- 172.21.15.50:0/2111047081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 msgr2=0x55946ef9f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.593+0000 7f97dc122740 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97c002a9e0 tx=0x7f97c0005720 comp rx=0 tx=0).stop 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- 172.21.15.50:0/2111047081 shutdown_connections 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebfd1b0 0x55946efb1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946ef9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- 172.21.15.50:0/2111047081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebd42f0 0x55946ef9f5e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- 172.21.15.50:0/2111047081 >> 172.21.15.50:0/2111047081 conn(0x55946ebf1300 msgr2=0x55946ef9eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- 172.21.15.50:0/2111047081 shutdown_connections 2024-05-17T04:42:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- 172.21.15.50:0/2111047081 wait complete. 2024-05-17T04:42:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- start start 2024-05-17T04:42:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 Processor -- start 2024-05-17T04:42:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 0x55946efb1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efaedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01aa710 con 0x55946ebfd1b0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01abb80 con 0x55946ebdc4d0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.594+0000 7f97dc122740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01aade0 con 0x55946ebd42f0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97dc122740 10 client.0 _ll_register_callbacks cb 0x7f97c80095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ce7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efaedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ce7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44526/0 (socket says 172.21.15.50:44526) 2024-05-17T04:42:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 0x55946efb1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ce7fc640 1 -- 172.21.15.50:0/2640148392 learned_addr learned my addr 172.21.15.50:0/2640148392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 -- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 msgr2=0x55946efae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 -- 172.21.15.50:0/2640148392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 msgr2=0x55946efaedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efaedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ceffd640 1 -- 172.21.15.50:0/2640148392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946f01da20 con 0x55946ebd42f0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97cf7fe640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efaedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.595+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97ceffd640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 0x55946efb1be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55946efa18b0 tx=0x7f97c002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a86a0d78f3a283f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c00315c0 con 0x55946ebd42f0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c0053070 con 0x55946ebd42f0 2024-05-17T04:42:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f97c005c070 con 0x55946ebd42f0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f97c006c050 con 0x55946ebd42f0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97dc122740 10 client.25504 fetch_fsmap learned FSMap version 38 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55946f01ec90 con 0x55946ebd42f0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.596+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97c0053070 con 0x55946ebd42f0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f97c005e070 con 0x55946ebd42f0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 10 client.25504 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 20 client.25504 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946eb911c0 0x55946f0211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 4 client.25504 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55946ef0e3f0 con 0x55946eb911c0 2024-05-17T04:42:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946eb911c0 0x55946f0211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97b2ff5640 10 client.25504 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.597+0000 7f97ce7fc640 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946eb911c0 0x55946f0211b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca5ea646ee7f00f server_cookie=3f8d42c82802abf2 in_seq=0 out_seq=0 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.611+0000 7f97b2ff5640 1 -- 172.21.15.50:0/2640148392 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55946ef0e3f0 con 0x55946eb911c0 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.611+0000 7f97b2ff5640 10 client.25504 handle_command_reply: tid=0 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.611+0000 7f97dc122740 1 client.25504 shutdown 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 msgr2=0x55946efb1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 0x55946efb1be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55946efa18b0 tx=0x7f97c002a3f0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 shutdown_connections 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55946eb911c0 0x55946f0211b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97b2ff5640 0 client.25504 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55946ebfd1b0 0x55946efaedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55946ebdc4d0 0x55946efae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 --2- 172.21.15.50:0/2640148392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55946ebd42f0 0x55946efb1be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 >> 172.21.15.50:0/2640148392 conn(0x55946ebf1300 msgr2=0x55946efa3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 shutdown_connections 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 1 -- 172.21.15.50:0/2640148392 wait complete. 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.612+0000 7f97dc122740 20 client.25504 trim_cache size 0 max 16384 2024-05-17T04:42:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:42:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:42:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:42:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 msgr2=0x7f97b80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 0x7f97b80b4d70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55946efa1960 tx=0x7f97c80097e0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 msgr2=0x7f97d0199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97c400b4a0 tx=0x7f97c400b960 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 shutdown_connections 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97d0103af0 0x7f97d01a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d01031a0 0x7f97d0199e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f97b80b2860 0x7f97b80b4d70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 --2- 172.21.15.50:0/3198014156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97d0102080 0x7f97d01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 >> 172.21.15.50:0/3198014156 conn(0x7f97d00fd880 msgr2=0x7f97d00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 shutdown_connections 2024-05-17T04:42:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:03.613+0000 7f97d55cb640 1 -- 172.21.15.50:0/3198014156 wait complete. 2024-05-17T04:42:04.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:04 smithi105 ceph-mon[34795]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-17T04:42:04.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:04 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:04.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:04 smithi110 ceph-mon[35497]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-17T04:42:04.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:04 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:04 smithi050 ceph-mon[31739]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-17T04:42:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:04 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:06 smithi105 ceph-mon[34795]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:06.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:06 smithi110 ceph-mon[35497]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:06 smithi050 ceph-mon[31739]: pgmap v612: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.0 MiB/s wr, 339 op/s 2024-05-17T04:42:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 190.0 s, 132.8 tps, lat 749.337 ms stddev 240.821 2024-05-17T04:42:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:08 smithi105 ceph-mon[34795]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-05-17T04:42:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:08 smithi110 ceph-mon[35497]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-05-17T04:42:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:08 smithi050 ceph-mon[31739]: pgmap v613: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 315 op/s 2024-05-17T04:42:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:10 smithi105 ceph-mon[34795]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-05-17T04:42:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:10 smithi110 ceph-mon[35497]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-05-17T04:42:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:10 smithi050 ceph-mon[31739]: pgmap v614: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-05-17T04:42:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:12 smithi105 ceph-mon[34795]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:12 smithi110 ceph-mon[35497]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:12.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:12 smithi050 ceph-mon[31739]: pgmap v615: 129 pgs: 129 active+clean; 1.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 195.0 s, 126.6 tps, lat 776.280 ms stddev 186.701 2024-05-17T04:42:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:14 smithi105 ceph-mon[34795]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-05-17T04:42:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:14 smithi110 ceph-mon[35497]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-05-17T04:42:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:14 smithi050 ceph-mon[31739]: pgmap v616: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 406 op/s 2024-05-17T04:42:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:16 smithi105 ceph-mon[34795]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:16.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:16 smithi110 ceph-mon[35497]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:16 smithi050 ceph-mon[31739]: pgmap v617: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-05-17T04:42:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 -- 172.21.15.50:0/2445673495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc01062e0 msgr2=0x7f2cc0108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 --2- 172.21.15.50:0/2445673495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc01062e0 0x7f2cc0108780 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009900 tx=0x7f2cb402a410 comp rx=0 tx=0).stop 2024-05-17T04:42:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 -- 172.21.15.50:0/2445673495 shutdown_connections 2024-05-17T04:42:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 --2- 172.21.15.50:0/2445673495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc0108e40 0x7f2cc010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 --2- 172.21.15.50:0/2445673495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc01062e0 0x7f2cc0108780 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 --2- 172.21.15.50:0/2445673495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.739+0000 7f2cc597f640 1 -- 172.21.15.50:0/2445673495 >> 172.21.15.50:0/2445673495 conn(0x7f2cc00fd8b0 msgr2=0x7f2cc00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.740+0000 7f2cc597f640 1 -- 172.21.15.50:0/2445673495 shutdown_connections 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.740+0000 7f2cc597f640 1 -- 172.21.15.50:0/2445673495 wait complete. 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.740+0000 7f2cc597f640 1 Processor -- start 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 -- start start 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 0x7f2cc0101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc0108e40 0x7f2cc0102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010ebc0 con 0x7f2cc0108e40 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010ea40 con 0x7f2cc006bbd0 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cc597f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc010ed40 con 0x7f2cc01062e0 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc0108e40 0x7f2cc0102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54336/0 (socket says 172.21.15.50:54336) 2024-05-17T04:42:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 -- 172.21.15.50:0/3569369821 learned_addr learned my addr 172.21.15.50:0/3569369821 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 0x7f2cc0101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 -- 172.21.15.50:0/3569369821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 msgr2=0x7f2cc0101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 0x7f2cc0101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 -- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc0108e40 msgr2=0x7f2cc0102150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc0108e40 0x7f2cc0102150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbeffd640 1 -- 172.21.15.50:0/3569369821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc0102880 con 0x7f2cc006bbd0 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.741+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 0x7f2cc0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e9c0 tx=0x7f2cb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cc497d640 1 -- 172.21.15.50:0/3569369821 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000cd80 con 0x7f2cc006bbd0 2024-05-17T04:42:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cc497d640 1 -- 172.21.15.50:0/3569369821 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb00044b0 con 0x7f2cc006bbd0 2024-05-17T04:42:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cc497d640 1 -- 172.21.15.50:0/3569369821 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0010430 con 0x7f2cc006bbd0 2024-05-17T04:42:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01a3880 con 0x7f2cc006bbd0 2024-05-17T04:42:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.742+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01a3cf0 con 0x7f2cc006bbd0 2024-05-17T04:42:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.745+0000 7f2ccc83f740 1 Processor -- start 2024-05-17T04:42:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.745+0000 7f2cc497d640 1 -- 172.21.15.50:0/3569369821 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2cb0003f00 con 0x7f2cc006bbd0 2024-05-17T04:42:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 -- start start 2024-05-17T04:42:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e872742f0 0x559e87640cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e87647c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e875b2cc0 con 0x559e8727c4d0 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e875a2860 con 0x559e872742f0 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e875a68a0 con 0x559e8729d1b0 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59630/0 (socket says 172.21.15.50:59630) 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3057042234 learned_addr learned my addr 172.21.15.50:0/3057042234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e872742f0 0x559e87640cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.746+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e87647c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3057042234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e872742f0 msgr2=0x559e87640cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e872742f0 0x559e87640cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3057042234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 msgr2=0x559e87647c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e87647c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3057042234 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e8756ad50 con 0x559e8729d1b0 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cc497d640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 0x7f2ca80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009fd0 tx=0x7f2cb80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a1d6108f5405d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 0x7f2ca80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2cc497d640 1 -- 172.21.15.50:0/3569369821 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2cb00e2aa0 con 0x7f2cc006bbd0 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2ca5ffb640 1 -- 172.21.15.50:0/3057042234 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004d20 con 0x559e8729d1b0 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2ca5ffb640 1 -- 172.21.15.50:0/3057042234 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb8004050 con 0x559e8729d1b0 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.747+0000 7f2ca5ffb640 1 -- 172.21.15.50:0/3057042234 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8005240 con 0x559e8729d1b0 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.748+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3057042234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 msgr2=0x559e87640770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.748+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009fd0 tx=0x7f2cb80118d0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.748+0000 7f2cbeffd640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 0x7f2ca80b4d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00a9c60 tx=0x7f2cb00e5f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3057042234 shutdown_connections 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e87647c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e872742f0 0x559e87640cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3057042234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8729d1b0 0x559e87640770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3057042234 >> 172.21.15.50:0/3057042234 conn(0x559e87291300 msgr2=0x559e8763c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3057042234 shutdown_connections 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3057042234 wait complete. 2024-05-17T04:42:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.749+0000 7f2ccc83f740 1 -- start start 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 Processor -- start 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8729d1b0 0x559e876514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e874deab0 con 0x559e8727c4d0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e8755f4d0 con 0x559e8729d1b0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e87648690 con 0x559e872742f0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2ccc83f740 10 client.0 _ll_register_callbacks cb 0x559e8759d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59634/0 (socket says 172.21.15.50:59634) 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54620/0 (socket says 172.21.15.50:54620) 2024-05-17T04:42:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/3975667247 learned_addr learned my addr 172.21.15.50:0/3975667247 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.750+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8729d1b0 0x559e876514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8729d1b0 msgr2=0x559e876514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8729d1b0 0x559e876514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3975667247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 msgr2=0x559e8764a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3975667247 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e87560340 con 0x559e872742f0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559e87641960 tx=0x7f2cb8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c509a55ee84893 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2c9effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8004a40 con 0x559e872742f0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2cb8004be0 con 0x559e872742f0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.751+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb8004050 con 0x559e872742f0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8036050 con 0x559e872742f0 2024-05-17T04:42:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 10 client.25516 fetch_fsmap learned FSMap version 38 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559e87308790 con 0x559e872742f0 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb8004050 con 0x559e872742f0 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f2cb8028450 con 0x559e872742f0 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 10 client.25516 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 20 client.25516 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8725bd80 0x559e87657510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 4 client.25516 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.752+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e875b4c10 con 0x559e8725bd80 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.753+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8725bd80 0x559e87657510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.753+0000 7f2c9effd640 10 client.25516 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.754+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8725bd80 0x559e87657510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1e64dd826950bb server_cookie=5aca2ab7b250f8c in_seq=0 out_seq=0 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.754+0000 7f2c9effd640 1 -- 172.21.15.50:0/3975667247 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e875b4c10 con 0x559e8725bd80 2024-05-17T04:42:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.754+0000 7f2c9effd640 10 client.25516 handle_command_reply: tid=0 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.754+0000 7f2ccc83f740 1 client.25516 shutdown 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 msgr2=0x559e8764a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559e87641960 tx=0x7f2cb8000f40 comp rx=0 tx=0).stop 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 shutdown_connections 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8729d1b0 0x559e876514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8725bd80 0x559e87657510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e8727c4d0 0x559e8764a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3975667247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e872742f0 0x559e8764a030 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2c9effd640 0 client.25516 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 >> 172.21.15.50:0/3975667247 conn(0x559e87291300 msgr2=0x559e8763b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 shutdown_connections 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3975667247 wait complete. 2024-05-17T04:42:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.755+0000 7f2ccc83f740 20 client.25516 trim_cache size 0 max 16384 2024-05-17T04:42:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.775+0000 7f2ccc83f740 1 Processor -- start 2024-05-17T04:42:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 -- start start 2024-05-17T04:42:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8763aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01b0cc0 con 0x559e872742f0 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01ae560 con 0x559e8725bd80 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01ac3b0 con 0x559e8727c4d0 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54634/0 (socket says 172.21.15.50:54634) 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/1810027602 learned_addr learned my addr 172.21.15.50:0/1810027602 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8763aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/1810027602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 msgr2=0x559e876519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/1810027602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 msgr2=0x559e8763aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8763aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/1810027602 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e87381620 con 0x559e872742f0 2024-05-17T04:42:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.776+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.777+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40057c0 tx=0x7f2cb4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a90ab9a35a41440 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.777+0000 7f2ca57fa640 1 -- 172.21.15.50:0/1810027602 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4035040 con 0x559e872742f0 2024-05-17T04:42:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.777+0000 7f2ca57fa640 1 -- 172.21.15.50:0/1810027602 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb404b070 con 0x559e872742f0 2024-05-17T04:42:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.777+0000 7f2ca57fa640 1 -- 172.21.15.50:0/1810027602 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4003dd0 con 0x559e872742f0 2024-05-17T04:42:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- 172.21.15.50:0/1810027602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 msgr2=0x559e87651460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40057c0 tx=0x7f2cb4005a20 comp rx=0 tx=0).stop 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- 172.21.15.50:0/1810027602 shutdown_connections 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e87651460 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/1810027602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8763aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- 172.21.15.50:0/1810027602 >> 172.21.15.50:0/1810027602 conn(0x559e87291300 msgr2=0x559e8763a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- 172.21.15.50:0/1810027602 shutdown_connections 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- 172.21.15.50:0/1810027602 wait complete. 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.778+0000 7f2ccc83f740 1 -- start start 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 Processor -- start 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8764e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e8764ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876bd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01aca20 con 0x559e872742f0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01ade90 con 0x559e8725bd80 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc01ad0f0 con 0x559e8727c4d0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2ccc83f740 10 client.0 _ll_register_callbacks cb 0x7f2cb80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e8764ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2cbe7fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e8764ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54640/0 (socket says 172.21.15.50:54640) 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2cbe7fc640 1 -- 172.21.15.50:0/3988911018 learned_addr learned my addr 172.21.15.50:0/3988911018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2cbf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8764e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.779+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3988911018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 msgr2=0x559e876bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3988911018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 msgr2=0x559e8764ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e8764ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 -- 172.21.15.50:0/3988911018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e87651ef0 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2cbf7fe640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8764e9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005d70 tx=0x7f2cb8009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2c6d7a8adeaaab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2c9effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8028660 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb404b070 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2cb800f4e0 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8004070 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2ccc83f740 10 client.25529 fetch_fsmap learned FSMap version 38 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559e876bed10 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.780+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cb404b070 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f2cb80045e0 con 0x559e8725bd80 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 10 client.25529 fetch_fsmap finished waiting for FSMap version 38 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 10 client.25529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 20 client.25529 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8729d1b0 0x559e876c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 4 client.25529 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:42:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e875ae3f0 con 0x559e8729d1b0 2024-05-17T04:42:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8729d1b0 0x559e876c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.781+0000 7f2c9effd640 10 client.25529 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.782+0000 7f2cbe7fc640 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8729d1b0 0x559e876c16a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e2a499f9c22ed1 server_cookie=d9df124020d5b81e in_seq=0 out_seq=0 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2c9effd640 1 -- 172.21.15.50:0/3988911018 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e875ae3f0 con 0x559e8729d1b0 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2c9effd640 10 client.25529 handle_command_reply: tid=0 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 client.25529 shutdown 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 msgr2=0x559e8764e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8764e9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005d70 tx=0x7f2cb8009d00 comp rx=0 tx=0).stop 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 shutdown_connections 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e8727c4d0 0x559e876bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559e8729d1b0 0x559e876c16a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e872742f0 0x559e8764ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 --2- 172.21.15.50:0/3988911018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e8725bd80 0x559e8764e9c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2c9effd640 0 client.25529 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 >> 172.21.15.50:0/3988911018 conn(0x559e87291300 msgr2=0x559e8763eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 shutdown_connections 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.804+0000 7f2ccc83f740 1 -- 172.21.15.50:0/3988911018 wait complete. 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2ccc83f740 20 client.25529 trim_cache size 0 max 16384 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:42:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 msgr2=0x7f2ca80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 0x7f2ca80b4d70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00a9c60 tx=0x7f2cb00e5f90 comp rx=0 tx=0).stop 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 msgr2=0x7f2cc0103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000e9c0 tx=0x7f2cb000ee80 comp rx=0 tx=0).stop 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 shutdown_connections 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ca80b2860 0x7f2ca80b4d70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cc0108e40 0x7f2cc0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2cc01062e0 0x7f2cc0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 --2- 172.21.15.50:0/3569369821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cc006bbd0 0x7f2cc0103570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 >> 172.21.15.50:0/3569369821 conn(0x7f2cc00fd8b0 msgr2=0x7f2cc01044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 shutdown_connections 2024-05-17T04:42:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:16.805+0000 7f2cc597f640 1 -- 172.21.15.50:0/3569369821 wait complete. 2024-05-17T04:42:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 200.0 s, 143.6 tps, lat 707.732 ms stddev 175.923 2024-05-17T04:42:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:18 smithi105 ceph-mon[34795]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-05-17T04:42:18.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:18 smithi110 ceph-mon[35497]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-05-17T04:42:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:18 smithi050 ceph-mon[31739]: pgmap v618: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-05-17T04:42:19.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:19 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:19.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:19 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:19.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:19 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:42:20.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:20 smithi050 ceph-mon[31739]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-05-17T04:42:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:20 smithi105 ceph-mon[34795]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-05-17T04:42:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:20 smithi110 ceph-mon[35497]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-05-17T04:42:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:22 smithi105 ceph-mon[34795]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.1 MiB/s wr, 360 op/s 2024-05-17T04:42:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:22 smithi110 ceph-mon[35497]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.1 MiB/s wr, 360 op/s 2024-05-17T04:42:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:22 smithi050 ceph-mon[31739]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.1 MiB/s wr, 360 op/s 2024-05-17T04:42:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 205.0 s, 122.2 tps, lat 812.866 ms stddev 225.339 2024-05-17T04:42:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:24 smithi105 ceph-mon[34795]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 364 op/s 2024-05-17T04:42:24.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:24 smithi110 ceph-mon[35497]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 364 op/s 2024-05-17T04:42:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:24 smithi050 ceph-mon[31739]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 364 op/s 2024-05-17T04:42:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:26 smithi105 ceph-mon[34795]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-17T04:42:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:26 smithi110 ceph-mon[35497]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-17T04:42:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:26 smithi050 ceph-mon[31739]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 328 op/s 2024-05-17T04:42:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 210.0 s, 147.4 tps, lat 684.134 ms stddev 179.314 2024-05-17T04:42:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:28 smithi105 ceph-mon[34795]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 374 op/s 2024-05-17T04:42:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:28 smithi110 ceph-mon[35497]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 374 op/s 2024-05-17T04:42:28.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:28 smithi050 ceph-mon[31739]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 374 op/s 2024-05-17T04:42:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 -- 172.21.15.50:0/3693403748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0102090 msgr2=0x7fecb01024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 --2- 172.21.15.50:0/3693403748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0102090 0x7fecb01024a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feca0009900 tx=0x7feca002a410 comp rx=0 tx=0).stop 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 -- 172.21.15.50:0/3693403748 shutdown_connections 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 --2- 172.21.15.50:0/3693403748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb0103b00 0x7fecb010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 --2- 172.21.15.50:0/3693403748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb01031b0 0x7fecb01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 --2- 172.21.15.50:0/3693403748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0102090 0x7fecb01024a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 -- 172.21.15.50:0/3693403748 >> 172.21.15.50:0/3693403748 conn(0x7fecb00fd8b0 msgr2=0x7fecb00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 -- 172.21.15.50:0/3693403748 shutdown_connections 2024-05-17T04:42:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.951+0000 7fecb6368640 1 -- 172.21.15.50:0/3693403748 wait complete. 2024-05-17T04:42:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.952+0000 7fecb6368640 1 Processor -- start 2024-05-17T04:42:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.952+0000 7fecb6368640 1 -- start start 2024-05-17T04:42:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0103b00 0x7fecb01a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57412/0 (socket says 172.21.15.50:57412) 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39578/0 (socket says 172.21.15.50:39578) 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecaffff640 1 -- 172.21.15.50:0/4158858590 learned_addr learned my addr 172.21.15.50:0/4158858590 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0110dc0 con 0x7fecb0102090 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0110c40 con 0x7fecb0103b00 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0110f40 con 0x7fecb01031b0 2024-05-17T04:42:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.953+0000 7fecb4905640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0103b00 0x7fecb01a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 -- 172.21.15.50:0/4158858590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 msgr2=0x7fecb019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 -- 172.21.15.50:0/4158858590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0103b00 msgr2=0x7fecb01a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0103b00 0x7fecb01a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01a3530 con 0x7fecb0102090 2024-05-17T04:42:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecaffff640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feca0005760 tx=0x7feca0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecad7fa640 1 -- 172.21.15.50:0/4158858590 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca003f070 con 0x7fecb0102090 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb01a37c0 con 0x7fecb0102090 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecad7fa640 1 -- 172.21.15.50:0/4158858590 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feca0031ba0 con 0x7fecb0102090 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.954+0000 7fecad7fa640 1 -- 172.21.15.50:0/4158858590 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0049060 con 0x7fecb0102090 2024-05-17T04:42:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.955+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb01a3d00 con 0x7fecb0102090 2024-05-17T04:42:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.958+0000 7fecbd225740 1 Processor -- start 2024-05-17T04:42:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.958+0000 7fecbd225740 1 -- start start 2024-05-17T04:42:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1dd11b0 0x5575a2174770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2174cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5575a20e6cc0 con 0x5575a1dd11b0 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5575a20d6860 con 0x5575a1da82f0 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecbd225740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575a20da8a0 con 0x5575a1db04d0 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecad7fa640 1 -- 172.21.15.50:0/4158858590 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feca0042070 con 0x7fecb0102090 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecb4905640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1dd11b0 0x5575a2174770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57420/0 (socket says 172.21.15.50:57420) 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecaf7fe640 1 -- 172.21.15.50:0/1616525345 learned_addr learned my addr 172.21.15.50:0/1616525345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.959+0000 7fecaffff640 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2174cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 -- 172.21.15.50:0/1616525345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 msgr2=0x5575a217bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 -- 172.21.15.50:0/1616525345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1dd11b0 msgr2=0x5575a2174770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1dd11b0 0x5575a2174770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 -- 172.21.15.50:0/1616525345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a209ed50 con 0x5575a1da82f0 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fecaffff640 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2174cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feca0100100 tx=0x7feca0108c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254461e1c3db5935 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.960+0000 7fec967fc640 1 -- 172.21.15.50:0/1616525345 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca010c670 con 0x5575a1da82f0 2024-05-17T04:42:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.961+0000 7fecad7fa640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 0x7fec980b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.961+0000 7fec967fc640 1 -- 172.21.15.50:0/1616525345 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feca0112020 con 0x5575a1da82f0 2024-05-17T04:42:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.961+0000 7fec967fc640 1 -- 172.21.15.50:0/1616525345 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca010dd00 con 0x5575a1da82f0 2024-05-17T04:42:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.961+0000 7fecb4905640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 0x7fec980b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.961+0000 7fecad7fa640 1 -- 172.21.15.50:0/4158858590 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feca01029c0 con 0x7fecb0102090 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.962+0000 7fecbd225740 1 -- 172.21.15.50:0/1616525345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 msgr2=0x5575a2174cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.962+0000 7fecb4905640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 0x7fec980b4d60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5575a2175960 tx=0x7feca8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.962+0000 7fecbd225740 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2174cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feca0100100 tx=0x7feca0108c80 comp rx=0 tx=0).stop 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 -- 172.21.15.50:0/1616525345 shutdown_connections 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1db04d0 0x5575a217bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2174cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 --2- 172.21.15.50:0/1616525345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1dd11b0 0x5575a2174770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 -- 172.21.15.50:0/1616525345 >> 172.21.15.50:0/1616525345 conn(0x5575a1dc5300 msgr2=0x5575a21701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 -- 172.21.15.50:0/1616525345 shutdown_connections 2024-05-17T04:42:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 -- 172.21.15.50:0/1616525345 wait complete. 2024-05-17T04:42:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.963+0000 7fecbd225740 1 -- start start 2024-05-17T04:42:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecbd225740 1 Processor -- start 2024-05-17T04:42:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecbd225740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 0x5575a217dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecbd225740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 0x5575a217e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 0x5575a217dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecaffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 0x5575a217dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39598/0 (socket says 172.21.15.50:39598) 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecaffff640 1 -- 172.21.15.50:0/1361400702 learned_addr learned my addr 172.21.15.50:0/1361400702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2185220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.964+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5575a2012ab0 con 0x5575a1da82f0 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5575a20934d0 con 0x5575a1db04d0 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5575a217c450 con 0x5575a1dd11b0 2024-05-17T04:42:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 0x5575a217e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecbd225740 10 client.0 _ll_register_callbacks cb 0x5575a20d1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecb4905640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2185220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 -- 172.21.15.50:0/1361400702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 msgr2=0x5575a2185220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2185220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 -- 172.21.15.50:0/1361400702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 msgr2=0x5575a217dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 0x5575a217dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a2094340 con 0x5575a1db04d0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fec8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 0x5575a217e2c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feca4012cd0 tx=0x7feca400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fde85fabce24824d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.965+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca40120d0 con 0x5575a1db04d0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feca40047a0 con 0x5575a1db04d0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca0112020 con 0x5575a1db04d0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feca4012270 con 0x5575a1db04d0 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fecbd225740 10 client.25541 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5575a1e3c790 con 0x5575a1db04d0 2024-05-17T04:42:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feca4004e30 con 0x5575a1db04d0 2024-05-17T04:42:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.966+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7feca401c470 con 0x5575a1db04d0 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 10 client.25541 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 20 client.25541 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d8fd80 0x5575a218b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 4 client.25541 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feca0112020 con 0x5575a1d8fd80 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fecaffff640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d8fd80 0x5575a218b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.967+0000 7fec8f7fe640 10 client.25541 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.968+0000 7fecaffff640 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d8fd80 0x5575a218b620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323f4e436b59e0da server_cookie=92ab9fbb385b80b1 in_seq=0 out_seq=0 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fec8f7fe640 1 -- 172.21.15.50:0/1361400702 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7feca0112020 con 0x5575a1d8fd80 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fec8f7fe640 10 client.25541 handle_command_reply: tid=0 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 client.25541 shutdown 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 msgr2=0x5575a217e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 0x5575a217e2c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feca4012cd0 tx=0x7feca400f790 comp rx=0 tx=0).stop 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 shutdown_connections 2024-05-17T04:42:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2185220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1db04d0 0x5575a217e2c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d8fd80 0x5575a218b620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 --2- 172.21.15.50:0/1361400702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1da82f0 0x5575a217dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fec8f7fe640 0 client.25541 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.969+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 >> 172.21.15.50:0/1361400702 conn(0x5575a1dc5300 msgr2=0x5575a2173010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.970+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 shutdown_connections 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.970+0000 7fecbd225740 1 -- 172.21.15.50:0/1361400702 wait complete. 2024-05-17T04:42:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.970+0000 7fecbd225740 20 client.25541 trim_cache size 0 max 16384 2024-05-17T04:42:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 Processor -- start 2024-05-17T04:42:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 -- start start 2024-05-17T04:42:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2185140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1dd11b0 0x5575a2185680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01b09e0 con 0x5575a1db04d0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01ae2b0 con 0x5575a1dd11b0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecbd225740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01ac100 con 0x5575a1da82f0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecaffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecaf7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2185140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecaffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57444/0 (socket says 172.21.15.50:57444) 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecaffff640 1 -- 172.21.15.50:0/1679369075 learned_addr learned my addr 172.21.15.50:0/1679369075 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.993+0000 7fecb4905640 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1dd11b0 0x5575a2185680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 -- 172.21.15.50:0/1679369075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1dd11b0 msgr2=0x5575a2185680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1dd11b0 0x5575a2185680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 -- 172.21.15.50:0/1679369075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 msgr2=0x5575a2185140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2185140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 -- 172.21.15.50:0/1679369075 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a1eb5620 con 0x5575a1da82f0 2024-05-17T04:42:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fecaffff640 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feca01004a0 tx=0x7feca0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b7047aac334c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fec95ffb640 1 -- 172.21.15.50:0/1679369075 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0111020 con 0x5575a1da82f0 2024-05-17T04:42:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fec95ffb640 1 -- 172.21.15.50:0/1679369075 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feca00d23f0 con 0x5575a1da82f0 2024-05-17T04:42:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.994+0000 7fec95ffb640 1 -- 172.21.15.50:0/1679369075 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00c9ad0 con 0x5575a1da82f0 2024-05-17T04:42:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.995+0000 7fecbd225740 1 -- 172.21.15.50:0/1679369075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 msgr2=0x5575a2177b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.995+0000 7fecbd225740 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feca01004a0 tx=0x7feca0009260 comp rx=0 tx=0).stop 2024-05-17T04:42:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 -- 172.21.15.50:0/1679369075 shutdown_connections 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1dd11b0 0x5575a2185680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2185140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 --2- 172.21.15.50:0/1679369075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1da82f0 0x5575a2177b80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 -- 172.21.15.50:0/1679369075 >> 172.21.15.50:0/1679369075 conn(0x5575a1dc5300 msgr2=0x5575a216f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 -- 172.21.15.50:0/1679369075 shutdown_connections 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 -- 172.21.15.50:0/1679369075 wait complete. 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 -- start start 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.996+0000 7fecbd225740 1 Processor -- start 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2183110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2184040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01ac770 con 0x5575a1db04d0 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01adbe0 con 0x5575a1da82f0 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fecb01ace40 con 0x5575a1dd11b0 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecbd225740 10 client.0 _ll_register_callbacks cb 0x7feca4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37760/0 (socket says 172.21.15.50:37760) 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 -- 172.21.15.50:0/952831418 learned_addr learned my addr 172.21.15.50:0/952831418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecb4905640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2184040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2183110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 -- 172.21.15.50:0/952831418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 msgr2=0x5575a2184040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2184040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 -- 172.21.15.50:0/952831418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 msgr2=0x5575a2183110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2183110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaffff640 1 -- 172.21.15.50:0/952831418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a2184630 con 0x5575a1da82f0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecb4905640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2184040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.997+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2183110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecaffff640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feca00c9a00 tx=0x7feca002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7a68215eadaa3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0111070 con 0x5575a1da82f0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feca003b460 con 0x5575a1da82f0 2024-05-17T04:42:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575a20e8c10 con 0x5575a1da82f0 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feca00c9470 con 0x5575a1da82f0 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecbd225740 10 client.25547 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5575a21858c0 con 0x5575a1da82f0 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5575a20e8c10 con 0x5575a1da82f0 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7feca004c060 con 0x5575a1da82f0 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecbd225740 10 client.25547 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.998+0000 7fecbd225740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.999+0000 7fecbd225740 20 client.25547 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.999+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d651c0 0x5575a21f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.999+0000 7fecbd225740 4 client.25547 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:42:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.999+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575a20e23f0 con 0x5575a1d651c0 2024-05-17T04:42:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:29.999+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d651c0 0x5575a21f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.000+0000 7fecaf7fe640 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d651c0 0x5575a21f59a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec9577b6df7dd6e server_cookie=232aec23a13f1422 in_seq=0 out_seq=0 2024-05-17T04:42:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.000+0000 7fec8f7fe640 10 client.25547 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.018+0000 7fec8f7fe640 1 -- 172.21.15.50:0/952831418 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575a20e23f0 con 0x5575a1d651c0 2024-05-17T04:42:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.018+0000 7fec8f7fe640 10 client.25547 handle_command_reply: tid=0 2024-05-17T04:42:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.018+0000 7fecbd225740 1 client.25547 shutdown 2024-05-17T04:42:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 msgr2=0x5575a2182bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feca00c9a00 tx=0x7feca002a410 comp rx=0 tx=0).stop 2024-05-17T04:42:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 shutdown_connections 2024-05-17T04:42:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5575a1dd11b0 0x5575a2184040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5575a1d651c0 0x5575a21f59a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5575a1db04d0 0x5575a2183110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 --2- 172.21.15.50:0/952831418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575a1da82f0 0x5575a2182bd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 >> 172.21.15.50:0/952831418 conn(0x5575a1dc5300 msgr2=0x5575a2172b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fec8f7fe640 0 client.25547 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 shutdown_connections 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 1 -- 172.21.15.50:0/952831418 wait complete. 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.019+0000 7fecbd225740 20 client.25547 trim_cache size 0 max 16384 2024-05-17T04:42:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:42:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:42:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:42:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 msgr2=0x7fec980b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 0x7fec980b4d60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5575a2175960 tx=0x7feca8009500 comp rx=0 tx=0).stop 2024-05-17T04:42:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 msgr2=0x7fecb019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7feca0005760 tx=0x7feca0003c10 comp rx=0 tx=0).stop 2024-05-17T04:42:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 shutdown_connections 2024-05-17T04:42:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0103b00 0x7fecb01a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fecb01031b0 0x7fecb019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fec980b2850 0x7fec980b4d60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 --2- 172.21.15.50:0/4158858590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecb0102090 0x7fecb019b9f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 >> 172.21.15.50:0/4158858590 conn(0x7fecb00fd8b0 msgr2=0x7fecb01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 shutdown_connections 2024-05-17T04:42:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:30.020+0000 7fecb6368640 1 -- 172.21.15.50:0/4158858590 wait complete. 2024-05-17T04:42:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:30 smithi105 ceph-mon[34795]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:42:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:30 smithi110 ceph-mon[35497]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:42:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:30 smithi050 ceph-mon[31739]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:42:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:32 smithi105 ceph-mon[34795]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-05-17T04:42:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:32 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:42:32.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:32 smithi110 ceph-mon[35497]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-05-17T04:42:32.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:32 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:42:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:32 smithi050 ceph-mon[31739]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.3 MiB/s wr, 337 op/s 2024-05-17T04:42:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:32 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:42:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 215.0 s, 135.0 tps, lat 742.510 ms stddev 247.384 2024-05-17T04:42:33.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:42:33.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:42:33.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:33 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:42:33.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:42:33.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:42:33.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:33 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:42:33.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:42:33.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:42:33.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:33 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:42:34.465 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:34 smithi105 ceph-mon[34795]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-17T04:42:34.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:34 smithi110 ceph-mon[35497]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-17T04:42:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:34 smithi050 ceph-mon[31739]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 344 op/s 2024-05-17T04:42:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:36 smithi105 ceph-mon[34795]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-17T04:42:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:36 smithi110 ceph-mon[35497]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-17T04:42:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:36 smithi050 ceph-mon[31739]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 366 op/s 2024-05-17T04:42:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 220.0 s, 157.6 tps, lat 629.843 ms stddev 178.702 2024-05-17T04:42:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:38 smithi105 ceph-mon[34795]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-05-17T04:42:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:38 smithi110 ceph-mon[35497]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-05-17T04:42:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:38 smithi050 ceph-mon[31739]: pgmap v628: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-05-17T04:42:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:40 smithi105 ceph-mon[34795]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-05-17T04:42:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:40 smithi110 ceph-mon[35497]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-05-17T04:42:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:40 smithi050 ceph-mon[31739]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 3.9 MiB/s wr, 314 op/s 2024-05-17T04:42:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:42 smithi105 ceph-mon[34795]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.9 MiB/s wr, 343 op/s 2024-05-17T04:42:42.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:42 smithi110 ceph-mon[35497]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.9 MiB/s wr, 343 op/s 2024-05-17T04:42:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:42 smithi050 ceph-mon[31739]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.9 MiB/s wr, 343 op/s 2024-05-17T04:42:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 225.0 s, 151.6 tps, lat 653.565 ms stddev 192.909 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 -- 172.21.15.50:0/1166435459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90102930 msgr2=0x7fcb90104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 --2- 172.21.15.50:0/1166435459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90102930 0x7fcb90104d50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009900 tx=0x7fcb8002a410 comp rx=0 tx=0).stop 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 -- 172.21.15.50:0/1166435459 shutdown_connections 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 --2- 172.21.15.50:0/1166435459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90107d20 0x7fcb9010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 --2- 172.21.15.50:0/1166435459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb901076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 --2- 172.21.15.50:0/1166435459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90102930 0x7fcb90104d50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.151+0000 7fcb967da640 1 -- 172.21.15.50:0/1166435459 >> 172.21.15.50:0/1166435459 conn(0x7fcb900fcbb0 msgr2=0x7fcb900ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.152+0000 7fcb967da640 1 -- 172.21.15.50:0/1166435459 shutdown_connections 2024-05-17T04:42:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.152+0000 7fcb967da640 1 -- 172.21.15.50:0/1166435459 wait complete. 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.152+0000 7fcb967da640 1 Processor -- start 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 -- start start 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90107d20 0x7fcb9019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010c950 con 0x7fcb90107d20 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010c7d0 con 0x7fcb90102930 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb967da640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010cad0 con 0x7fcb90105290 2024-05-17T04:42:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58952/0 (socket says 172.21.15.50:58952) 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb8ffff640 1 -- 172.21.15.50:0/2805034182 learned_addr learned my addr 172.21.15.50:0/2805034182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41034/0 (socket says 172.21.15.50:41034) 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.153+0000 7fcb94d77640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90107d20 0x7fcb9019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 -- 172.21.15.50:0/2805034182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 msgr2=0x7fcb90197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 -- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90107d20 msgr2=0x7fcb9019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90107d20 0x7fcb9019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 -- 172.21.15.50:0/2805034182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9019f130 con 0x7fcb90102930 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006010 tx=0x7fcb80003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8d7fa640 1 -- 172.21.15.50:0/2805034182 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8003f070 con 0x7fcb90102930 2024-05-17T04:42:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9019f3c0 con 0x7fcb90102930 2024-05-17T04:42:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8d7fa640 1 -- 172.21.15.50:0/2805034182 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb80031c50 con 0x7fcb90102930 2024-05-17T04:42:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.154+0000 7fcb8d7fa640 1 -- 172.21.15.50:0/2805034182 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80035030 con 0x7fcb90102930 2024-05-17T04:42:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.155+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9019f900 con 0x7fcb90102930 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 Processor -- start 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb8d7fa640 1 -- 172.21.15.50:0/2805034182 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcb80042070 con 0x7fcb90102930 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 -- start start 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b211b0 0x55a409ec4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ec4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a409e36cc0 con 0x55a409af82f0 2024-05-17T04:42:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a409e26860 con 0x55a409b211b0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb9d768740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a409e2a8a0 con 0x55a409b004d0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ec4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.158+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb94d77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b211b0 0x55a409ec4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ec4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33242/0 (socket says 172.21.15.50:33242) 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41044/0 (socket says 172.21.15.50:41044) 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8ffff640 1 -- 172.21.15.50:0/2664213969 learned_addr learned my addr 172.21.15.50:0/2664213969 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/2664213969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b211b0 msgr2=0x55a409ec4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b211b0 0x55a409ec4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/2664213969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 msgr2=0x55a409ec4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ec4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/2664213969 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a409deed50 con 0x55a409b004d0 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8d7fa640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 0x7fcb780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009990 tx=0x7fcb84009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0df4be944845c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.159+0000 7fcb94d77640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 0x7fcb780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.160+0000 7fcb8d7fa640 1 -- 172.21.15.50:0/2805034182 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcb80102a00 con 0x7fcb90102930 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.160+0000 7fcb767fc640 1 -- 172.21.15.50:0/2664213969 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84004230 con 0x55a409b004d0 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.160+0000 7fcb767fc640 1 -- 172.21.15.50:0/2664213969 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb840043d0 con 0x55a409b004d0 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.160+0000 7fcb767fc640 1 -- 172.21.15.50:0/2664213969 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84003d40 con 0x55a409b004d0 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.161+0000 7fcb9d768740 1 -- 172.21.15.50:0/2664213969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 msgr2=0x55a409ecbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.161+0000 7fcb9d768740 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009990 tx=0x7fcb84009e50 comp rx=0 tx=0).stop 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.161+0000 7fcb94d77640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 0x7fcb780b4cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a409ec5960 tx=0x7fcb8800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 -- 172.21.15.50:0/2664213969 shutdown_connections 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409b004d0 0x55a409ecbc10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ec4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 --2- 172.21.15.50:0/2664213969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b211b0 0x55a409ec4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 -- 172.21.15.50:0/2664213969 >> 172.21.15.50:0/2664213969 conn(0x55a409b15300 msgr2=0x55a409ec01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 -- 172.21.15.50:0/2664213969 shutdown_connections 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 -- 172.21.15.50:0/2664213969 wait complete. 2024-05-17T04:42:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.162+0000 7fcb9d768740 1 -- start start 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 Processor -- start 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409b211b0 0x55a409ed54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a409d62ab0 con 0x55a409b211b0 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a409de34d0 con 0x55a409b004d0 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a409ecc690 con 0x55a409af82f0 2024-05-17T04:42:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb9d768740 10 client.0 _ll_register_callbacks cb 0x55a409e21950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.163+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41054/0 (socket says 172.21.15.50:41054) 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8ffff640 1 -- 172.21.15.50:0/4099806847 learned_addr learned my addr 172.21.15.50:0/4099806847 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58968/0 (socket says 172.21.15.50:58968) 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb94d77640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409b211b0 0x55a409ed54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/4099806847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 msgr2=0x55a409ece030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/4099806847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409b211b0 msgr2=0x55a409ed54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409b211b0 0x55a409ed54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/4099806847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a409de4340 con 0x55a409b004d0 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009990 tx=0x7fcb84003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71e14b8134f9255 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.164+0000 7fcb6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84005450 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb840043d0 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcb840055f0 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb8403c050 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb9d768740 10 client.25559 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a409b8c790 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.165+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb840043d0 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcb84003e20 con 0x55a409b004d0 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 10 client.25559 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 20 client.25559 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409adfd80 0x55a409edb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 4 client.25559 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:42:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a409e38c10 con 0x55a409adfd80 2024-05-17T04:42:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409adfd80 0x55a409edb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.166+0000 7fcb6f7fe640 10 client.25559 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.167+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409adfd80 0x55a409edb470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee2c63558fd0a13 server_cookie=3eb4d8e52b44e676 in_seq=0 out_seq=0 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.167+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/4099806847 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a409e38c10 con 0x55a409adfd80 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.167+0000 7fcb6f7fe640 10 client.25559 handle_command_reply: tid=0 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 client.25559 shutdown 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 msgr2=0x55a409ece570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009990 tx=0x7fcb84003820 comp rx=0 tx=0).stop 2024-05-17T04:42:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 shutdown_connections 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409adfd80 0x55a409edb470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409b211b0 0x55a409ed54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb6f7fe640 0 client.25559 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ece570 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 --2- 172.21.15.50:0/4099806847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409af82f0 0x55a409ece030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 >> 172.21.15.50:0/4099806847 conn(0x55a409b15300 msgr2=0x55a409ec3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 shutdown_connections 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 1 -- 172.21.15.50:0/4099806847 wait complete. 2024-05-17T04:42:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.168+0000 7fcb9d768740 20 client.25559 trim_cache size 0 max 16384 2024-05-17T04:42:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.193+0000 7fcb9d768740 1 Processor -- start 2024-05-17T04:42:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.193+0000 7fcb9d768740 1 -- start start 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ebeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ed53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901ac6b0 con 0x55a409af82f0 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901a9f80 con 0x55a409b004d0 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb9d768740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901a7dd0 con 0x55a409adfd80 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ebeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33254/0 (socket says 172.21.15.50:33254) 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/1164831660 learned_addr learned my addr 172.21.15.50:0/1164831660 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb94d77640 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ed53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/1164831660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 msgr2=0x55a409ebeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ebeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/1164831660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 msgr2=0x55a409ed53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ed53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.194+0000 7fcb8f7fe640 1 -- 172.21.15.50:0/1164831660 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a409c05620 con 0x55a409af82f0 2024-05-17T04:42:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.195+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009960 tx=0x7fcb84033560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b13f458e9f2e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.195+0000 7fcb75ffb640 1 -- 172.21.15.50:0/1164831660 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84004390 con 0x55a409af82f0 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.195+0000 7fcb75ffb640 1 -- 172.21.15.50:0/1164831660 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb8402e070 con 0x55a409af82f0 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.195+0000 7fcb75ffb640 1 -- 172.21.15.50:0/1164831660 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84033e60 con 0x55a409af82f0 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 -- 172.21.15.50:0/1164831660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 msgr2=0x55a409ebf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009960 tx=0x7fcb84033560 comp rx=0 tx=0).stop 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 -- 172.21.15.50:0/1164831660 shutdown_connections 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409ed53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ebf430 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 --2- 172.21.15.50:0/1164831660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ebeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 -- 172.21.15.50:0/1164831660 >> 172.21.15.50:0/1164831660 conn(0x55a409b15300 msgr2=0x55a409ebe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.196+0000 7fcb9d768740 1 -- 172.21.15.50:0/1164831660 shutdown_connections 2024-05-17T04:42:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.197+0000 7fcb9d768740 1 -- 172.21.15.50:0/1164831660 wait complete. 2024-05-17T04:42:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.197+0000 7fcb9d768740 1 -- start start 2024-05-17T04:42:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 Processor -- start 2024-05-17T04:42:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ed2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ed3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901a8440 con 0x55a409af82f0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901a98b0 con 0x55a409b004d0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb9d768740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb901a8b10 con 0x55a409adfd80 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.198+0000 7fcb94d77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58988/0 (socket says 172.21.15.50:58988) 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 -- 172.21.15.50:0/956954916 learned_addr learned my addr 172.21.15.50:0/956954916 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ed2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ed3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb9d768740 10 client.0 _ll_register_callbacks cb 0x7fcb8010a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 -- 172.21.15.50:0/956954916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 msgr2=0x55a409ed2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ed2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 -- 172.21.15.50:0/956954916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 msgr2=0x55a409ed3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ed3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 -- 172.21.15.50:0/956954916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a409f41cc0 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb8f7fe640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ed3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb94d77640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcb880046c0 tx=0x7fcb88004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f47eed57a40480 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.199+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8802d070 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb8402e070 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcb88009040 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb88039050 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 10 client.25565 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a409f43050 con 0x55a409b004d0 2024-05-17T04:42:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb8402e070 con 0x55a409b004d0 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcb88029bd0 con 0x55a409b004d0 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 10 client.25565 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 20 client.25565 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409b211b0 0x55a409f459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 4 client.25565 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.200+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a409e323f0 con 0x55a409b211b0 2024-05-17T04:42:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.201+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409b211b0 0x55a409f459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.201+0000 7fcb6f7fe640 10 client.25565 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.201+0000 7fcb8ffff640 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409b211b0 0x55a409f459b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2509930a1b03482e server_cookie=152dd74a946bb269 in_seq=0 out_seq=0 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.217+0000 7fcb6f7fe640 1 -- 172.21.15.50:0/956954916 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a409e323f0 con 0x55a409b211b0 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.217+0000 7fcb6f7fe640 10 client.25565 handle_command_reply: tid=0 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.217+0000 7fcb9d768740 1 client.25565 shutdown 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 msgr2=0x55a409f41730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcb880046c0 tx=0x7fcb88004ab0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 shutdown_connections 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a409b004d0 0x55a409f41730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a409b211b0 0x55a409f459b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a409af82f0 0x55a409ed3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 --2- 172.21.15.50:0/956954916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a409adfd80 0x55a409ed2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb6f7fe640 0 client.25565 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 >> 172.21.15.50:0/956954916 conn(0x55a409b15300 msgr2=0x55a409ebf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 shutdown_connections 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 1 -- 172.21.15.50:0/956954916 wait complete. 2024-05-17T04:42:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.218+0000 7fcb9d768740 20 client.25565 trim_cache size 0 max 16384 2024-05-17T04:42:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:42:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:42:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:42:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 msgr2=0x7fcb780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 0x7fcb780b4cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a409ec5960 tx=0x7fcb8800f040 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 msgr2=0x7fcb901975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006010 tx=0x7fcb80003b20 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 shutdown_connections 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fcb780b27e0 0x7fcb780b4cf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcb90107d20 0x7fcb9019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb90105290 0x7fcb90197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 --2- 172.21.15.50:0/2805034182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb90102930 0x7fcb901975f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 >> 172.21.15.50:0/2805034182 conn(0x7fcb900fcbb0 msgr2=0x7fcb90104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 shutdown_connections 2024-05-17T04:42:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:43.219+0000 7fcb967da640 1 -- 172.21.15.50:0/2805034182 wait complete. 2024-05-17T04:42:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:44 smithi105 ceph-mon[34795]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-05-17T04:42:44.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:44 smithi110 ceph-mon[35497]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-05-17T04:42:44.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:44 smithi050 ceph-mon[31739]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-05-17T04:42:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:46 smithi105 ceph-mon[34795]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.2 MiB/s wr, 326 op/s 2024-05-17T04:42:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:46 smithi110 ceph-mon[35497]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.2 MiB/s wr, 326 op/s 2024-05-17T04:42:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:46 smithi050 ceph-mon[31739]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 4.2 MiB/s wr, 326 op/s 2024-05-17T04:42:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 230.0 s, 113.0 tps, lat 867.352 ms stddev 256.008 2024-05-17T04:42:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:48 smithi105 ceph-mon[34795]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.7 MiB/s wr, 323 op/s 2024-05-17T04:42:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:48 smithi110 ceph-mon[35497]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.7 MiB/s wr, 323 op/s 2024-05-17T04:42:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:48 smithi050 ceph-mon[31739]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.7 MiB/s wr, 323 op/s 2024-05-17T04:42:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:50 smithi105 ceph-mon[34795]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-17T04:42:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:50 smithi110 ceph-mon[35497]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-17T04:42:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:50 smithi050 ceph-mon[31739]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-05-17T04:42:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:52 smithi105 ceph-mon[34795]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.7 MiB/s wr, 324 op/s 2024-05-17T04:42:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:52 smithi110 ceph-mon[35497]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.7 MiB/s wr, 324 op/s 2024-05-17T04:42:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:52 smithi050 ceph-mon[31739]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.7 MiB/s wr, 324 op/s 2024-05-17T04:42:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 235.0 s, 117.6 tps, lat 835.401 ms stddev 209.357 2024-05-17T04:42:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:54 smithi105 ceph-mon[34795]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-05-17T04:42:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:54 smithi110 ceph-mon[35497]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-05-17T04:42:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:54 smithi050 ceph-mon[31739]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.351+0000 7fab82bbb640 1 -- 172.21.15.50:0/1879980973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101ad0 msgr2=0x7fab7c10b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.351+0000 7fab82bbb640 1 --2- 172.21.15.50:0/1879980973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101ad0 0x7fab7c10b6f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fab74009f90 tx=0x7fab7402a640 comp rx=0 tx=0).stop 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 -- 172.21.15.50:0/1879980973 shutdown_connections 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 --2- 172.21.15.50:0/1879980973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101ad0 0x7fab7c10b6f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 --2- 172.21.15.50:0/1879980973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c101180 0x7fab7c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 --2- 172.21.15.50:0/1879980973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c104740 0x7fab7c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 -- 172.21.15.50:0/1879980973 >> 172.21.15.50:0/1879980973 conn(0x7fab7c0fd020 msgr2=0x7fab7c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 -- 172.21.15.50:0/1879980973 shutdown_connections 2024-05-17T04:42:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.352+0000 7fab82bbb640 1 -- 172.21.15.50:0/1879980973 wait complete. 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 Processor -- start 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 -- start start 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 0x7fab7c109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c104740 0x7fab7c105a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10eb80 con 0x7fab7c104740 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10ea00 con 0x7fab7c101180 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab82bbb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10ed00 con 0x7fab7c101ad0 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab80957640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab80957640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49430/0 (socket says 172.21.15.50:49430) 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab81158640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c104740 0x7fab7c105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab80957640 1 -- 172.21.15.50:0/4110091657 learned_addr learned my addr 172.21.15.50:0/4110091657 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.353+0000 7fab7bfff640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 0x7fab7c109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 -- 172.21.15.50:0/4110091657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 msgr2=0x7fab7c109430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 0x7fab7c109430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 -- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c104740 msgr2=0x7fab7c105a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c104740 0x7fab7c105a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 -- 172.21.15.50:0/4110091657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c106260 con 0x7fab7c101180 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab7bfff640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 0x7fab7c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab80957640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9c0 tx=0x7fab7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab79ffb640 1 -- 172.21.15.50:0/4110091657 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7000cd80 con 0x7fab7c101180 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab79ffb640 1 -- 172.21.15.50:0/4110091657 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab700054b0 con 0x7fab7c101180 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c106550 con 0x7fab7c101180 2024-05-17T04:42:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.354+0000 7fab79ffb640 1 -- 172.21.15.50:0/4110091657 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70010430 con 0x7fab7c101180 2024-05-17T04:42:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.355+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a80b0 con 0x7fab7c101180 2024-05-17T04:42:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.358+0000 7fab79ffb640 1 -- 172.21.15.50:0/4110091657 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fab70004f20 con 0x7fab7c101180 2024-05-17T04:42:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.359+0000 7fab89a7c740 1 Processor -- start 2024-05-17T04:42:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.359+0000 7fab89a7c740 1 -- start start 2024-05-17T04:42:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 0x5630061a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 0x5630061abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab81158640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 0x5630061a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab81158640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 0x5630061a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49444/0 (socket says 172.21.15.50:49444) 2024-05-17T04:42:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 0x5630061abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 0x5630061abc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60434/0 (socket says 172.21.15.50:60434) 2024-05-17T04:42:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab79ffb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 0x7fab6c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.360+0000 7fab81158640 1 -- 172.21.15.50:0/3821300087 learned_addr learned my addr 172.21.15.50:0/3821300087 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563006116cc0 con 0x563005de04d0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563006106860 con 0x563005e011b0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56300610a8a0 con 0x563005dd82f0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab79ffb640 1 -- 172.21.15.50:0/4110091657 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab700e2b20 con 0x7fab7c101180 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab81158640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 0x7fab6c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 -- 172.21.15.50:0/3821300087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 msgr2=0x5630061a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 0x5630061a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 -- 172.21.15.50:0/3821300087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 msgr2=0x5630061abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 0x5630061abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.361+0000 7fab80957640 1 -- 172.21.15.50:0/3821300087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630060ced50 con 0x563005dd82f0 2024-05-17T04:42:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.362+0000 7fab80957640 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061a4cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab700a6fc0 tx=0x7fab700a9760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18ccd22bfaea65f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.362+0000 7fab66ffd640 1 -- 172.21.15.50:0/3821300087 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700a9d50 con 0x563005dd82f0 2024-05-17T04:42:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.362+0000 7fab66ffd640 1 -- 172.21.15.50:0/3821300087 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab700ae100 con 0x563005dd82f0 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.362+0000 7fab81158640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 0x7fab6c0b4df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5630061a5960 tx=0x7fab7402a640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.362+0000 7fab66ffd640 1 -- 172.21.15.50:0/3821300087 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700a56c0 con 0x563005dd82f0 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.363+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 msgr2=0x5630061a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.363+0000 7fab89a7c740 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061a4cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab700a6fc0 tx=0x7fab700a9760 comp rx=0 tx=0).stop 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 shutdown_connections 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005de04d0 0x5630061abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061a4cf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 --2- 172.21.15.50:0/3821300087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005e011b0 0x5630061a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 >> 172.21.15.50:0/3821300087 conn(0x563005df5300 msgr2=0x5630061a01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 shutdown_connections 2024-05-17T04:42:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.364+0000 7fab89a7c740 1 -- 172.21.15.50:0/3821300087 wait complete. 2024-05-17T04:42:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.365+0000 7fab89a7c740 1 -- start start 2024-05-17T04:42:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.365+0000 7fab89a7c740 1 Processor -- start 2024-05-17T04:42:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.366+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x5630061ae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563006042ab0 con 0x563005e011b0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630060c34d0 con 0x563005dd82f0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630061ac630 con 0x563005de04d0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab89a7c740 10 client.0 _ll_register_callbacks cb 0x563006101950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab81158640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab81158640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60446/0 (socket says 172.21.15.50:60446) 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab81158640 1 -- 172.21.15.50:0/2474558832 learned_addr learned my addr 172.21.15.50:0/2474558832 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.367+0000 7fab80957640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 -- 172.21.15.50:0/2474558832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 msgr2=0x5630061ae570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x5630061ae570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 -- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 msgr2=0x5630061ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 -- 172.21.15.50:0/2474558832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630060c4340 con 0x563005e011b0 2024-05-17T04:42:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab80957640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:42:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab81158640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab74031b00 tx=0x7fab74009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835d8900d6272def server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:42:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74046050 con 0x563005e011b0 2024-05-17T04:42:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.368+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab700ae100 con 0x563005e011b0 2024-05-17T04:42:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.369+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab74008b20 con 0x563005e011b0 2024-05-17T04:42:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.369+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab74036040 con 0x563005e011b0 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.369+0000 7fab89a7c740 10 client.15813 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.369+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563005e6c790 con 0x563005e011b0 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.370+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab74050070 con 0x563005e011b0 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.370+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fab74039070 con 0x563005e011b0 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.370+0000 7fab89a7c740 10 client.15813 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.370+0000 7fab89a7c740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab89a7c740 20 client.15813 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005dbfd80 0x5630061bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab89a7c740 4 client.15813 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab700ae100 con 0x563005dbfd80 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab80957640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005dbfd80 0x5630061bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab5bfff640 10 client.15813 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.371+0000 7fab80957640 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005dbfd80 0x5630061bb9b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c496bf857798073b server_cookie=b2d18559e7d05ea1 in_seq=0 out_seq=0 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.372+0000 7fab5bfff640 1 -- 172.21.15.50:0/2474558832 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fab700ae100 con 0x563005dbfd80 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.372+0000 7fab5bfff640 10 client.15813 handle_command_reply: tid=0 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.372+0000 7fab89a7c740 1 client.15813 shutdown 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 msgr2=0x5630061b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab74031b00 tx=0x7fab74009920 comp rx=0 tx=0).stop 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 shutdown_connections 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005dbfd80 0x5630061bb9b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005e011b0 0x5630061b54d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab5bfff640 0 client.15813 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x5630061ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2474558832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.373+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 >> 172.21.15.50:0/2474558832 conn(0x563005df5300 msgr2=0x5630061a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.374+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 shutdown_connections 2024-05-17T04:42:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.374+0000 7fab89a7c740 1 -- 172.21.15.50:0/2474558832 wait complete. 2024-05-17T04:42:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.374+0000 7fab89a7c740 20 client.15813 trim_cache size 0 max 16384 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.393+0000 7fab89a7c740 1 Processor -- start 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.393+0000 7fab89a7c740 1 -- start start 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 0x5630061a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1ac6c0 con 0x563005dbfd80 2024-05-17T04:42:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1a9f90 con 0x563005de04d0 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab89a7c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1a3910 con 0x563005dd82f0 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60450/0 (socket says 172.21.15.50:60450) 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 -- 172.21.15.50:0/498788629 learned_addr learned my addr 172.21.15.50:0/498788629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab81158640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 0x5630061a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab7bfff640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 -- 172.21.15.50:0/498788629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 msgr2=0x5630061b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 -- 172.21.15.50:0/498788629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 msgr2=0x5630061a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 0x5630061a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab80957640 1 -- 172.21.15.50:0/498788629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563005ee5620 con 0x563005dbfd80 2024-05-17T04:42:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.394+0000 7fab81158640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 0x5630061a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:42:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.395+0000 7fab80957640 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fab70000c00 tx=0x7fab70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da777a704a3cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.395+0000 7fab667fc640 1 -- 172.21.15.50:0/498788629 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700ae100 con 0x563005dbfd80 2024-05-17T04:42:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.395+0000 7fab667fc640 1 -- 172.21.15.50:0/498788629 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab700a40c0 con 0x563005dbfd80 2024-05-17T04:42:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.395+0000 7fab667fc640 1 -- 172.21.15.50:0/498788629 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700a60d0 con 0x563005dbfd80 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- 172.21.15.50:0/498788629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 msgr2=0x5630061a8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fab70000c00 tx=0x7fab70005880 comp rx=0 tx=0).stop 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- 172.21.15.50:0/498788629 shutdown_connections 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005de04d0 0x5630061a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005dd82f0 0x5630061b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 --2- 172.21.15.50:0/498788629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061a8bc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- 172.21.15.50:0/498788629 >> 172.21.15.50:0/498788629 conn(0x563005df5300 msgr2=0x5630061a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- 172.21.15.50:0/498788629 shutdown_connections 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- 172.21.15.50:0/498788629 wait complete. 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 -- start start 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.396+0000 7fab89a7c740 1 Processor -- start 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x563006221fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60466/0 (socket says 172.21.15.50:60466) 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1a8450 con 0x563005dbfd80 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 -- 172.21.15.50:0/2076957228 learned_addr learned my addr 172.21.15.50:0/2076957228 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1a98c0 con 0x563005dd82f0 2024-05-17T04:42:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1a8b20 con 0x563005de04d0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab89a7c740 10 client.0 _ll_register_callbacks cb 0x7fab70005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab7bfff640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab81158640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x563006221fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 -- 172.21.15.50:0/2076957228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 msgr2=0x563006221fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x563006221fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 -- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 msgr2=0x5630061b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab80957640 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563006222620 con 0x563005dbfd80 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.397+0000 7fab7bfff640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab80957640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab700a6fc0 tx=0x7fab70004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22a7aa1dd0b6ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab700b2bb0 con 0x563005dbfd80 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fab700a6340 con 0x563005dbfd80 2024-05-17T04:42:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab700a40c0 con 0x563005dbfd80 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fab700a9d50 con 0x563005dbfd80 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 10 client.15819 fetch_fsmap learned FSMap version 39 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563006223890 con 0x563005dbfd80 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab700a40c0 con 0x563005dbfd80 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fab700c3020 con 0x563005dbfd80 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 10 client.15819 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 20 client.15819 populate_metadata read hostname 'smithi050' 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005e011b0 0x563006225ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 4 client.15819 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.398+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563006118c10 con 0x563005e011b0 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.399+0000 7fab7bfff640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005e011b0 0x563006225ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.399+0000 7fab5bfff640 10 client.15819 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.399+0000 7fab7bfff640 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005e011b0 0x563006225ca0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca55a5485058fed8 server_cookie=3684f1b94a208758 in_seq=0 out_seq=0 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab5bfff640 1 -- 172.21.15.50:0/2076957228 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563006118c10 con 0x563005e011b0 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab5bfff640 10 client.15819 handle_command_reply: tid=0 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 client.15819 shutdown 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 msgr2=0x5630061b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab700a6fc0 tx=0x7fab70004920 comp rx=0 tx=0).stop 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 shutdown_connections 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563005de04d0 0x563006221fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563005dd82f0 0x5630061b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563005e011b0 0x563006225ca0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 --2- 172.21.15.50:0/2076957228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563005dbfd80 0x5630061b4880 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab5bfff640 0 client.15819 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.413+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 >> 172.21.15.50:0/2076957228 conn(0x563005df5300 msgr2=0x5630061a7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 shutdown_connections 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab89a7c740 1 -- 172.21.15.50:0/2076957228 wait complete. 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab89a7c740 20 client.15819 trim_cache size 0 max 16384 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:42:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 msgr2=0x7fab6c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 0x7fab6c0b4df0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5630061a5960 tx=0x7fab7402a640 comp rx=0 tx=0).stop 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 msgr2=0x7fab7c108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.414+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fab7000e9c0 tx=0x7fab7000ee80 comp rx=0 tx=0).stop 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 shutdown_connections 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fab6c0b28e0 0x7fab6c0b4df0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab7c104740 0x7fab7c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab7c101ad0 0x7fab7c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 --2- 172.21.15.50:0/4110091657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab7c101180 0x7fab7c108ef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:42:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 >> 172.21.15.50:0/4110091657 conn(0x7fab7c0fd020 msgr2=0x7fab7c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:42:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 shutdown_connections 2024-05-17T04:42:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:42:56.415+0000 7fab82bbb640 1 -- 172.21.15.50:0/4110091657 wait complete. 2024-05-17T04:42:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:56 smithi105 ceph-mon[34795]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-05-17T04:42:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:56 smithi110 ceph-mon[35497]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-05-17T04:42:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:56 smithi050 ceph-mon[31739]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-05-17T04:42:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 240.0 s, 146.4 tps, lat 716.340 ms stddev 223.863 2024-05-17T04:42:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:42:58 smithi105 ceph-mon[34795]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-17T04:42:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:42:58 smithi110 ceph-mon[35497]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-17T04:42:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:42:58 smithi050 ceph-mon[31739]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.6 MiB/s wr, 310 op/s 2024-05-17T04:43:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:00 smithi105 ceph-mon[34795]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-05-17T04:43:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:00 smithi110 ceph-mon[35497]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-05-17T04:43:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:00 smithi050 ceph-mon[31739]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 3.9 MiB/s wr, 313 op/s 2024-05-17T04:43:02.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:02 smithi105 ceph-mon[34795]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-05-17T04:43:02.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:02 smithi110 ceph-mon[35497]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-05-17T04:43:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:02 smithi050 ceph-mon[31739]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-05-17T04:43:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 245.0 s, 126.6 tps, lat 775.437 ms stddev 257.109 2024-05-17T04:43:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:04 smithi105 ceph-mon[34795]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-05-17T04:43:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:04 smithi110 ceph-mon[35497]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-05-17T04:43:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:04 smithi050 ceph-mon[31739]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-05-17T04:43:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:06 smithi105 ceph-mon[34795]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-05-17T04:43:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:06 smithi110 ceph-mon[35497]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-05-17T04:43:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:06 smithi050 ceph-mon[31739]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-05-17T04:43:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 250.0 s, 151.4 tps, lat 670.060 ms stddev 211.891 2024-05-17T04:43:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:08 smithi105 ceph-mon[34795]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-05-17T04:43:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:08 smithi110 ceph-mon[35497]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-05-17T04:43:08.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:08 smithi050 ceph-mon[31739]: pgmap v643: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.6 MiB/s wr, 324 op/s 2024-05-17T04:43:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 -- 172.21.15.50:0/3453618671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee81042a0 msgr2=0x7f2ee81046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/3453618671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee81042a0 0x7f2ee81046b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009900 tx=0x7f2edc02a3f0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 -- 172.21.15.50:0/3453618671 shutdown_connections 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/3453618671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8104bf0 0x7f2ee810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/3453618671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee81042a0 0x7f2ee81046b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/3453618671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee8103180 0x7f2ee8103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 -- 172.21.15.50:0/3453618671 >> 172.21.15.50:0/3453618671 conn(0x7f2ee80fe9a0 msgr2=0x7f2ee8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 -- 172.21.15.50:0/3453618671 shutdown_connections 2024-05-17T04:43:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.549+0000 7f2eee2a3640 1 -- 172.21.15.50:0/3453618671 wait complete. 2024-05-17T04:43:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.550+0000 7f2eee2a3640 1 Processor -- start 2024-05-17T04:43:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.550+0000 7f2eee2a3640 1 -- start start 2024-05-17T04:43:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.550+0000 7f2eee2a3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.550+0000 7f2eee2a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 0x7f2ee8075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.550+0000 7f2eee2a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee8104bf0 0x7f2ee80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2eee2a3640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810eb60 con 0x7f2ee8103180 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2eee2a3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810e9e0 con 0x7f2ee8104bf0 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2eee2a3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810ece0 con 0x7f2ee81042a0 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 0x7f2ee8075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44202/0 (socket says 172.21.15.50:44202) 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee7fff640 1 -- 172.21.15.50:0/656394750 learned_addr learned my addr 172.21.15.50:0/656394750 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2eec840640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee8104bf0 0x7f2ee80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 -- 172.21.15.50:0/656394750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee8104bf0 msgr2=0x7f2ee80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee8104bf0 0x7f2ee80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 -- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 msgr2=0x7f2ee80769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 -- 172.21.15.50:0/656394750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee8075d00 con 0x7f2ee81042a0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.551+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 0x7f2ee8075090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005760 tx=0x7f2edc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.552+0000 7f2ee57fa640 1 -- 172.21.15.50:0/656394750 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc03f070 con 0x7f2ee81042a0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.552+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a38d0 con 0x7f2ee81042a0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.552+0000 7f2ee57fa640 1 -- 172.21.15.50:0/656394750 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2edc032e20 con 0x7f2ee81042a0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.552+0000 7f2ee57fa640 1 -- 172.21.15.50:0/656394750 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc035070 con 0x7f2ee81042a0 2024-05-17T04:43:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.552+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a3cd0 con 0x7f2ee81042a0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 Processor -- start 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 -- start start 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461804c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56246176fcc0 con 0x5624614312f0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56246175f860 con 0x56246145a1b0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ef5232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5624617638a0 con 0x5624614394d0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2eec840640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44214/0 (socket says 172.21.15.50:44214) 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ee7fff640 1 -- 172.21.15.50:0/2459909197 learned_addr learned my addr 172.21.15.50:0/2459909197 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.555+0000 7f2ee57fa640 1 -- 172.21.15.50:0/656394750 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2edc0037f0 con 0x7f2ee81042a0 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 -- 172.21.15.50:0/2459909197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 msgr2=0x562461804c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461804c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 -- 172.21.15.50:0/2459909197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 msgr2=0x5624617fdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 -- 172.21.15.50:0/2459909197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562461727d50 con 0x56246145a1b0 2024-05-17T04:43:09.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.556+0000 7f2eec840640 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fd770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009f00 tx=0x7f2ee0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f512a2f5912c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ece7fc640 1 -- 172.21.15.50:0/2459909197 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0004d20 con 0x56246145a1b0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ece7fc640 1 -- 172.21.15.50:0/2459909197 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ee0004030 con 0x56246145a1b0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ece7fc640 1 -- 172.21.15.50:0/2459909197 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0005260 con 0x56246145a1b0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ee57fa640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 0x7f2ed00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 0x7f2ed00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.557+0000 7f2ee57fa640 1 -- 172.21.15.50:0/656394750 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2edc102bc0 con 0x7f2ee81042a0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- 172.21.15.50:0/2459909197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 msgr2=0x5624617fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 0x7f2ed00b4cf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5624617feb70 tx=0x7f2ed800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fd770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009f00 tx=0x7f2ee0011810 comp rx=0 tx=0).stop 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- 172.21.15.50:0/2459909197 shutdown_connections 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461804c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x5624617fdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 --2- 172.21.15.50:0/2459909197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fd770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- 172.21.15.50:0/2459909197 >> 172.21.15.50:0/2459909197 conn(0x56246144e300 msgr2=0x5624617f91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- 172.21.15.50:0/2459909197 shutdown_connections 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- 172.21.15.50:0/2459909197 wait complete. 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.558+0000 7f2ef5232740 1 -- start start 2024-05-17T04:43:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 Processor -- start 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614312f0 0x562461807100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 0x562461807640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56246169bab0 con 0x56246145a1b0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56246171c4d0 con 0x5624614394d0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562461805690 con 0x5624614312f0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ef5232740 10 client.0 _ll_register_callbacks cb 0x56246175a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2eec840640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 0x562461807640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2eec840640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44216/0 (socket says 172.21.15.50:44216) 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2eec840640 1 -- 172.21.15.50:0/76516201 learned_addr learned my addr 172.21.15.50:0/76516201 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614312f0 0x562461807100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 -- 172.21.15.50:0/76516201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614312f0 msgr2=0x562461807100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614312f0 0x562461807100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 -- 172.21.15.50:0/76516201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 msgr2=0x56246180e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.559+0000 7f2ee77fe640 1 -- 172.21.15.50:0/76516201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246171d340 con 0x5624614394d0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 0x562461807640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2edc032fd0 tx=0x7f2edc0c9d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30643c3f47fa77b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc048040 con 0x5624614394d0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2edc0c6190 con 0x5624614394d0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee0004030 con 0x5624614394d0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2edc0c9a10 con 0x5624614394d0 2024-05-17T04:43:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ef5232740 10 client.25592 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5624614c5790 con 0x5624614394d0 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.560+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ee0004030 con 0x5624614394d0 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f2edc0e1060 con 0x5624614394d0 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 10 client.25592 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 20 client.25592 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562461418d80 0x562461814860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 4 client.25592 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562461771c10 con 0x562461418d80 2024-05-17T04:43:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.561+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562461418d80 0x562461814860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.562+0000 7f2ec77fe640 10 client.25592 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.562+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562461418d80 0x562461814860 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d32a3993a5beb02 server_cookie=ee9da60c258dea5d in_seq=0 out_seq=0 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ec77fe640 1 -- 172.21.15.50:0/76516201 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562461771c10 con 0x562461418d80 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ec77fe640 10 client.25592 handle_command_reply: tid=0 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 client.25592 shutdown 2024-05-17T04:43:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 msgr2=0x562461807640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 0x562461807640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2edc032fd0 tx=0x7f2edc0c9d50 comp rx=0 tx=0).stop 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 shutdown_connections 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562461418d80 0x562461814860 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614394d0 0x562461807640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 --2- 172.21.15.50:0/76516201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614312f0 0x562461807100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 >> 172.21.15.50:0/76516201 conn(0x56246144e300 msgr2=0x5624617f81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.563+0000 7f2ec77fe640 0 client.25592 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.564+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 shutdown_connections 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.564+0000 7f2ef5232740 1 -- 172.21.15.50:0/76516201 wait complete. 2024-05-17T04:43:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.564+0000 7f2ef5232740 20 client.25592 trim_cache size 0 max 16384 2024-05-17T04:43:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 Processor -- start 2024-05-17T04:43:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 -- start start 2024-05-17T04:43:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x5624617f9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81b0a60 con 0x5624614312f0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ae330 con 0x56246145a1b0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ef5232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ac180 con 0x5624614394d0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44218/0 (socket says 172.21.15.50:44218) 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ee7fff640 1 -- 172.21.15.50:0/4148333490 learned_addr learned my addr 172.21.15.50:0/4148333490 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2eec840640 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.585+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x5624617f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 -- 172.21.15.50:0/4148333490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 msgr2=0x5624617f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x5624617f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 -- 172.21.15.50:0/4148333490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 msgr2=0x5624617fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 -- 172.21.15.50:0/4148333490 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246153e620 con 0x5624614312f0 2024-05-17T04:43:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x562461806690 tx=0x7f2ed8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b0533fc996858 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ecdffb640 1 -- 172.21.15.50:0/4148333490 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed802f070 con 0x5624614312f0 2024-05-17T04:43:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ecdffb640 1 -- 172.21.15.50:0/4148333490 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed80043d0 con 0x5624614312f0 2024-05-17T04:43:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.586+0000 7f2ecdffb640 1 -- 172.21.15.50:0/4148333490 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8003a80 con 0x5624614312f0 2024-05-17T04:43:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 -- 172.21.15.50:0/4148333490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 msgr2=0x562461805cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x562461806690 tx=0x7f2ed8019000 comp rx=0 tx=0).stop 2024-05-17T04:43:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 -- 172.21.15.50:0/4148333490 shutdown_connections 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56246145a1b0 0x5624617fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x5624617f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 --2- 172.21.15.50:0/4148333490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5624614312f0 0x562461805cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 -- 172.21.15.50:0/4148333490 >> 172.21.15.50:0/4148333490 conn(0x56246144e300 msgr2=0x5624617f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 -- 172.21.15.50:0/4148333490 shutdown_connections 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.587+0000 7f2ef5232740 1 -- 172.21.15.50:0/4148333490 wait complete. 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 -- start start 2024-05-17T04:43:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 Processor -- start 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 0x562461808d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461809280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81ac7f0 con 0x56246145a1b0 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81adc60 con 0x5624614312f0 2024-05-17T04:43:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee81acec0 con 0x5624614394d0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ef5232740 10 client.0 _ll_register_callbacks cb 0x7f2edc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2eec840640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 0x562461808d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2eec840640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44230/0 (socket says 172.21.15.50:44230) 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2eec840640 1 -- 172.21.15.50:0/36680490 learned_addr learned my addr 172.21.15.50:0/36680490 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.588+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461809280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 -- 172.21.15.50:0/36680490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 msgr2=0x562461809280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461809280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 -- 172.21.15.50:0/36680490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 msgr2=0x56246180c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 -- 172.21.15.50:0/36680490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246180c7b0 con 0x5624614312f0 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2eec840640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461809280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ee7fff640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 0x562461808d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8011730 tx=0x7f2ed8010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb94444bb7f87a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8038020 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed80043d0 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.589+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2ed802a7a0 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2ed8043050 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 10 client.25598 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56246180da40 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ed80043d0 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed80344a0 con 0x5624614312f0 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 10 client.25598 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 20 client.25598 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5624613ee1c0 0x56246187ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 4 client.25598 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56246176b3f0 con 0x5624613ee1c0 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.590+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5624613ee1c0 0x56246187ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.591+0000 7f2ec77fe640 10 client.25598 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.591+0000 7f2ee77fe640 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5624613ee1c0 0x56246187ee30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9c7df9d9b6cb0c server_cookie=aa04aafa88da577f in_seq=0 out_seq=0 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ec77fe640 1 -- 172.21.15.50:0/36680490 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56246176b3f0 con 0x5624613ee1c0 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ec77fe640 10 client.25598 handle_command_reply: tid=0 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 client.25598 shutdown 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 msgr2=0x562461808d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 0x562461808d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8011730 tx=0x7f2ed8010fa0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 shutdown_connections 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5624613ee1c0 0x56246187ee30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56246145a1b0 0x56246180c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ec77fe640 0 client.25598 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5624614394d0 0x562461809280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 --2- 172.21.15.50:0/36680490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5624614312f0 0x562461808d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.610+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 >> 172.21.15.50:0/36680490 conn(0x56246144e300 msgr2=0x5624617f77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 shutdown_connections 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2ef5232740 1 -- 172.21.15.50:0/36680490 wait complete. 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2ef5232740 20 client.25598 trim_cache size 0 max 16384 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:43:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 msgr2=0x7f2ed00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 0x7f2ed00b4cf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5624617feb70 tx=0x7f2ed800b410 comp rx=0 tx=0).stop 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 msgr2=0x7f2ee8075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 0x7f2ee8075090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005760 tx=0x7f2edc004270 comp rx=0 tx=0).stop 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 shutdown_connections 2024-05-17T04:43:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee8104bf0 0x7f2ee80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ee81042a0 0x7f2ee8075090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2ed00b27e0 0x7f2ed00b4cf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 --2- 172.21.15.50:0/656394750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ee8103180 0x7f2ee80769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.611+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 >> 172.21.15.50:0/656394750 conn(0x7f2ee80fe9a0 msgr2=0x7f2ee81075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.612+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 shutdown_connections 2024-05-17T04:43:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:09.612+0000 7f2eee2a3640 1 -- 172.21.15.50:0/656394750 wait complete. 2024-05-17T04:43:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:10 smithi105 ceph-mon[34795]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-17T04:43:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:10 smithi110 ceph-mon[35497]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-17T04:43:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:10 smithi050 ceph-mon[31739]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-17T04:43:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:12 smithi105 ceph-mon[34795]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.4 MiB/s wr, 310 op/s 2024-05-17T04:43:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:12 smithi110 ceph-mon[35497]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.4 MiB/s wr, 310 op/s 2024-05-17T04:43:12.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:12 smithi050 ceph-mon[31739]: pgmap v645: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.4 MiB/s wr, 310 op/s 2024-05-17T04:43:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 255.0 s, 147.8 tps, lat 674.670 ms stddev 156.513 2024-05-17T04:43:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:14 smithi105 ceph-mon[34795]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-05-17T04:43:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:14 smithi110 ceph-mon[35497]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-05-17T04:43:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:14 smithi050 ceph-mon[31739]: pgmap v646: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.9 MiB/s wr, 384 op/s 2024-05-17T04:43:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:16 smithi105 ceph-mon[34795]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-17T04:43:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:16 smithi110 ceph-mon[35497]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-17T04:43:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:16 smithi050 ceph-mon[31739]: pgmap v647: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-17T04:43:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 260.0 s, 133.2 tps, lat 750.022 ms stddev 222.401 2024-05-17T04:43:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:18 smithi105 ceph-mon[34795]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 328 op/s 2024-05-17T04:43:18.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:18 smithi110 ceph-mon[35497]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 328 op/s 2024-05-17T04:43:18.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:18 smithi050 ceph-mon[31739]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 328 op/s 2024-05-17T04:43:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:20 smithi105 ceph-mon[34795]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-05-17T04:43:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:20 smithi110 ceph-mon[35497]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-05-17T04:43:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:20 smithi050 ceph-mon[31739]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-05-17T04:43:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:22 smithi105 ceph-mon[34795]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-05-17T04:43:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:22 smithi110 ceph-mon[35497]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-05-17T04:43:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:22 smithi050 ceph-mon[31739]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.6 MiB/s wr, 348 op/s 2024-05-17T04:43:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 -- 172.21.15.50:0/1843703818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd0103af0 msgr2=0x7f9cd010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 --2- 172.21.15.50:0/1843703818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd0103af0 0x7f9cd010a3f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009ca0 tx=0x7f9cc802a8c0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 -- 172.21.15.50:0/1843703818 shutdown_connections 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 --2- 172.21.15.50:0/1843703818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd0103af0 0x7f9cd010a3f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 --2- 172.21.15.50:0/1843703818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd01031a0 0x7f9cd01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 --2- 172.21.15.50:0/1843703818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 -- 172.21.15.50:0/1843703818 >> 172.21.15.50:0/1843703818 conn(0x7f9cd00fd880 msgr2=0x7f9cd00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 -- 172.21.15.50:0/1843703818 shutdown_connections 2024-05-17T04:43:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.749+0000 7f9cd523d640 1 -- 172.21.15.50:0/1843703818 wait complete. 2024-05-17T04:43:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 Processor -- start 2024-05-17T04:43:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 -- start start 2024-05-17T04:43:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd0103af0 0x7f9cd019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd010da10 con 0x7f9cd01031a0 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd010d890 con 0x7f9cd0102080 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cd523d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd010db90 con 0x7f9cd0103af0 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.750+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53760/0 (socket says 172.21.15.50:53760) 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58238/0 (socket says 172.21.15.50:58238) 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 -- 172.21.15.50:0/210502938 learned_addr learned my addr 172.21.15.50:0/210502938 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd0103af0 0x7f9cd019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 -- 172.21.15.50:0/210502938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd0103af0 msgr2=0x7f9cd019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd0103af0 0x7f9cd019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 -- 172.21.15.50:0/210502938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 msgr2=0x7f9cd01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 -- 172.21.15.50:0/210502938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd01a0200 con 0x7f9cd01031a0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cced9d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cccd99640 1 -- 172.21.15.50:0/210502938 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cd80 con 0x7f9cd01031a0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd01a04f0 con 0x7f9cd01031a0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cccd99640 1 -- 172.21.15.50:0/210502938 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cb80054b0 con 0x7f9cd01031a0 2024-05-17T04:43:22.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.751+0000 7f9cccd99640 1 -- 172.21.15.50:0/210502938 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8010430 con 0x7f9cd01031a0 2024-05-17T04:43:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.752+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd01a09f0 con 0x7f9cd01031a0 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.754+0000 7f9cdc1db740 1 Processor -- start 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 -- start start 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 0x5560d31b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 0x5560d31b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5560d312dcc0 con 0x5560d2def2f0 2024-05-17T04:43:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d311d860 con 0x5560d2df74d0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560d31218a0 con 0x5560d2e181b0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9ccf59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 0x5560d31b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53766/0 (socket says 172.21.15.50:53766) 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cced9d640 1 -- 172.21.15.50:0/2615039006 learned_addr learned my addr 172.21.15.50:0/2615039006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.755+0000 7f9cccd99640 1 -- 172.21.15.50:0/210502938 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9cb80046c0 con 0x7f9cd01031a0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 -- 172.21.15.50:0/2615039006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 msgr2=0x5560d31b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 0x5560d31b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 -- 172.21.15.50:0/2615039006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 msgr2=0x5560d31b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 0x5560d31b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 0x5560d31b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 -- 172.21.15.50:0/2615039006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d30e5d50 con 0x5560d2def2f0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 0x5560d31b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 0x5560d31b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cced9d640 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00114f0 tx=0x7f9cc00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce70a9fb53961f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.756+0000 7f9cc57fa640 1 -- 172.21.15.50:0/2615039006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001e070 con 0x5560d2def2f0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.757+0000 7f9cccd99640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 0x7f9cbc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.757+0000 7f9cc57fa640 1 -- 172.21.15.50:0/2615039006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cc00044c0 con 0x5560d2def2f0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.757+0000 7f9cc57fa640 1 -- 172.21.15.50:0/2615039006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000f820 con 0x5560d2def2f0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.757+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 0x7f9cbc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.757+0000 7f9cccd99640 1 -- 172.21.15.50:0/210502938 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9cb80e2910 con 0x7f9cd01031a0 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- 172.21.15.50:0/2615039006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 msgr2=0x5560d31b6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00114f0 tx=0x7f9cc00119b0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 0x7f9cbc0b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5560d31b8030 tx=0x7f9cc8031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- 172.21.15.50:0/2615039006 shutdown_connections 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2df74d0 0x5560d31b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31b6bf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/2615039006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2e181b0 0x5560d31b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- 172.21.15.50:0/2615039006 >> 172.21.15.50:0/2615039006 conn(0x5560d2e0c300 msgr2=0x5560d31b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- 172.21.15.50:0/2615039006 shutdown_connections 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- 172.21.15.50:0/2615039006 wait complete. 2024-05-17T04:43:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 -- start start 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.758+0000 7f9cdc1db740 1 Processor -- start 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 0x5560d31c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2e181b0 0x5560d31d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5560d3059ab0 con 0x5560d2df74d0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d30da4d0 con 0x5560d2e181b0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560d31c7ad0 con 0x5560d2def2f0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cdc1db740 10 client.0 _ll_register_callbacks cb 0x5560d3118950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48546/0 (socket says 172.21.15.50:48546) 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 -- 172.21.15.50:0/745455140 learned_addr learned my addr 172.21.15.50:0/745455140 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 0x5560d31c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2e181b0 0x5560d31d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 -- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2e181b0 msgr2=0x5560d31d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2e181b0 0x5560d31d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 -- 172.21.15.50:0/745455140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 msgr2=0x5560d31c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 0x5560d31c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 -- 172.21.15.50:0/745455140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d30db340 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 0x5560d31c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cced9d640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00114c0 tx=0x7f9cc0004030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37aa43bbb7e184be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.759+0000 7f9cb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001e040 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9cc0004220 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560d312fc10 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9cc0032050 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 10 client.25600 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5560d2e83790 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5560d312fc10 con 0x5560d2def2f0 2024-05-17T04:43:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9cc0021030 con 0x5560d2def2f0 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 10 client.25600 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 20 client.25600 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.760+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2dd6d80 0x5560d31d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.761+0000 7f9cdc1db740 4 client.25600 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.761+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d31293f0 con 0x5560d2dd6d80 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.761+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2dd6d80 0x5560d31d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.761+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2dd6d80 0x5560d31d6990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1fabceca13ed92 server_cookie=2d3ce83c3e72870b in_seq=0 out_seq=0 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.761+0000 7f9cb67fc640 10 client.25600 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cb67fc640 1 -- 172.21.15.50:0/745455140 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560d31293f0 con 0x5560d2dd6d80 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cb67fc640 10 client.25600 handle_command_reply: tid=0 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 client.25600 shutdown 2024-05-17T04:43:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 msgr2=0x5560d31c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00114c0 tx=0x7f9cc0004030 comp rx=0 tx=0).stop 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 shutdown_connections 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2e181b0 0x5560d31d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2dd6d80 0x5560d31d6990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2df74d0 0x5560d31c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cb67fc640 0 client.25600 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/745455140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2def2f0 0x5560d31c94d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 >> 172.21.15.50:0/745455140 conn(0x5560d2e0c300 msgr2=0x5560d31b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 shutdown_connections 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 1 -- 172.21.15.50:0/745455140 wait complete. 2024-05-17T04:43:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.762+0000 7f9cdc1db740 20 client.25600 trim_cache size 0 max 16384 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 Processor -- start 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 -- start start 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01ac6d0 con 0x5560d2def2f0 2024-05-17T04:43:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01a9fa0 con 0x5560d2dd6d80 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01a7df0 con 0x5560d2df74d0 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53780/0 (socket says 172.21.15.50:53780) 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9ccf59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cc7fff640 1 -- 172.21.15.50:0/3619777786 learned_addr learned my addr 172.21.15.50:0/3619777786 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.786+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58266/0 (socket says 172.21.15.50:58266) 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 -- 172.21.15.50:0/3619777786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 msgr2=0x5560d31d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 -- 172.21.15.50:0/3619777786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 msgr2=0x5560d31c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 -- 172.21.15.50:0/3619777786 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d2efc620 con 0x5560d2def2f0 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cced9d640 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a6e10 tx=0x7f9cb80cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d74e48f94c083f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc4ff9640 1 -- 172.21.15.50:0/3619777786 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80c3070 con 0x5560d2def2f0 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc4ff9640 1 -- 172.21.15.50:0/3619777786 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cb80cae50 con 0x5560d2def2f0 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.787+0000 7f9cc4ff9640 1 -- 172.21.15.50:0/3619777786 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80adf50 con 0x5560d2def2f0 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 -- 172.21.15.50:0/3619777786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 msgr2=0x5560d31d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a6e10 tx=0x7f9cb80cc040 comp rx=0 tx=0).stop 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 -- 172.21.15.50:0/3619777786 shutdown_connections 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2def2f0 0x5560d31d0900 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/3619777786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2dd6d80 0x5560d31c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 -- 172.21.15.50:0/3619777786 >> 172.21.15.50:0/3619777786 conn(0x5560d2e0c300 msgr2=0x5560d31c3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 -- 172.21.15.50:0/3619777786 shutdown_connections 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.788+0000 7f9cdc1db740 1 -- 172.21.15.50:0/3619777786 wait complete. 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 -- start start 2024-05-17T04:43:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 Processor -- start 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01a8460 con 0x5560d2dd6d80 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01a98d0 con 0x5560d2def2f0 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd01a8b30 con 0x5560d2df74d0 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cdc1db740 10 client.0 _ll_register_callbacks cb 0x7f9cc001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53796/0 (socket says 172.21.15.50:53796) 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cc7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:58268/0 (socket says 172.21.15.50:58268) 2024-05-17T04:43:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.789+0000 7f9cced9d640 1 -- 172.21.15.50:0/1522352910 learned_addr learned my addr 172.21.15.50:0/1522352910 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9ccf59e640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 -- 172.21.15.50:0/1522352910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 msgr2=0x5560d31cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 -- 172.21.15.50:0/1522352910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 msgr2=0x5560d31d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 -- 172.21.15.50:0/1522352910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d31ce460 con 0x5560d2def2f0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cced9d640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cc7fff640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5560d31ca200 tx=0x7f9cb80c4260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d0e7c6d7013d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80c3070 con 0x5560d2def2f0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9cb80b2780 con 0x5560d2def2f0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.790+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb80cae50 con 0x5560d2def2f0 2024-05-17T04:43:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9cb80adf50 con 0x5560d2def2f0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 10 client.25613 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5560d31bd5e0 con 0x5560d2def2f0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cb80cae50 con 0x5560d2def2f0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9cb80a5510 con 0x5560d2def2f0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 10 client.25613 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 20 client.25613 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2e181b0 0x5560d31bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 4 client.25613 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.791+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560d30a3750 con 0x5560d2e181b0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.792+0000 7f9cced9d640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2e181b0 0x5560d31bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.792+0000 7f9cb67fc640 10 client.25613 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.792+0000 7f9cced9d640 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2e181b0 0x5560d31bffa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43694be25e792ad server_cookie=8a73700917b55729 in_seq=0 out_seq=0 2024-05-17T04:43:22.800 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 265.0 s, 131.2 tps, lat 764.472 ms stddev 173.267 2024-05-17T04:43:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.810+0000 7f9cb67fc640 1 -- 172.21.15.50:0/1522352910 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560d30a3750 con 0x5560d2e181b0 2024-05-17T04:43:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cb67fc640 10 client.25613 handle_command_reply: tid=0 2024-05-17T04:43:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 client.25613 shutdown 2024-05-17T04:43:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 msgr2=0x5560d31cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5560d31ca200 tx=0x7f9cb80c4260 comp rx=0 tx=0).stop 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 shutdown_connections 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560d2df74d0 0x5560d31cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d2def2f0 0x5560d31cd990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5560d2e181b0 0x5560d31bffa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 --2- 172.21.15.50:0/1522352910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560d2dd6d80 0x5560d31d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 >> 172.21.15.50:0/1522352910 conn(0x5560d2e0c300 msgr2=0x5560d31b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cb67fc640 0 client.25613 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 shutdown_connections 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 1 -- 172.21.15.50:0/1522352910 wait complete. 2024-05-17T04:43:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.811+0000 7f9cdc1db740 20 client.25613 trim_cache size 0 max 16384 2024-05-17T04:43:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:43:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:43:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:43:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 msgr2=0x7f9cbc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 0x7f9cbc0b4cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5560d31b8030 tx=0x7f9cc8031000 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 msgr2=0x7f9cd0198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800edf0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 shutdown_connections 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cd0103af0 0x7f9cd019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9cbc0b27e0 0x7f9cbc0b4cf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9cd01031a0 0x7f9cd0198c00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 --2- 172.21.15.50:0/210502938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd0102080 0x7f9cd01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 >> 172.21.15.50:0/210502938 conn(0x7f9cd00fd880 msgr2=0x7f9cd0106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 shutdown_connections 2024-05-17T04:43:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:22.812+0000 7f9cd523d640 1 -- 172.21.15.50:0/210502938 wait complete. 2024-05-17T04:43:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:24 smithi105 ceph-mon[34795]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-05-17T04:43:24.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:24 smithi110 ceph-mon[35497]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-05-17T04:43:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:24 smithi050 ceph-mon[31739]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-05-17T04:43:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:26 smithi105 ceph-mon[34795]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-17T04:43:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:26 smithi110 ceph-mon[35497]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-17T04:43:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:26 smithi050 ceph-mon[31739]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 4.5 MiB/s wr, 345 op/s 2024-05-17T04:43:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 270.0 s, 135.2 tps, lat 741.166 ms stddev 172.380 2024-05-17T04:43:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:28 smithi105 ceph-mon[34795]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.3 MiB/s wr, 330 op/s 2024-05-17T04:43:28.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:28 smithi110 ceph-mon[35497]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.3 MiB/s wr, 330 op/s 2024-05-17T04:43:28.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:28 smithi050 ceph-mon[31739]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.3 MiB/s wr, 330 op/s 2024-05-17T04:43:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:30 smithi105 ceph-mon[34795]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-05-17T04:43:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:30 smithi110 ceph-mon[35497]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-05-17T04:43:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:30 smithi050 ceph-mon[31739]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-05-17T04:43:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:32 smithi105 ceph-mon[34795]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-05-17T04:43:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:32 smithi110 ceph-mon[35497]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-05-17T04:43:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:32 smithi050 ceph-mon[31739]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.8 MiB/s wr, 350 op/s 2024-05-17T04:43:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 275.0 s, 125.8 tps, lat 714.561 ms stddev 200.975 2024-05-17T04:43:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:34 smithi105 ceph-mon[34795]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-05-17T04:43:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:43:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:43:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:43:34.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:34 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:43:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:34 smithi110 ceph-mon[35497]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-05-17T04:43:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:43:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:43:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:43:34.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:34 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:43:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:34 smithi050 ceph-mon[31739]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.6 MiB/s wr, 404 op/s 2024-05-17T04:43:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:43:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:43:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:43:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:34 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:43:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 -- 172.21.15.50:0/434537127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c103550 msgr2=0x7fb59c103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 --2- 172.21.15.50:0/434537127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c103550 0x7fb59c103960 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb584009900 tx=0x7fb58402a410 comp rx=0 tx=0).stop 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 -- 172.21.15.50:0/434537127 shutdown_connections 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 --2- 172.21.15.50:0/434537127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103ea0 0x7fb59c10a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 --2- 172.21.15.50:0/434537127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c103550 0x7fb59c103960 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 --2- 172.21.15.50:0/434537127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c102660 0x7fb59c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 -- 172.21.15.50:0/434537127 >> 172.21.15.50:0/434537127 conn(0x7fb59c0fd010 msgr2=0x7fb59c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 -- 172.21.15.50:0/434537127 shutdown_connections 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.937+0000 7fb5a0865640 1 -- 172.21.15.50:0/434537127 wait complete. 2024-05-17T04:43:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 Processor -- start 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 -- start start 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c102660 0x7fb59c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c103ea0 0x7fb59c1a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c10dc90 con 0x7fb59c102660 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c10db10 con 0x7fb59c103550 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb5a0865640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c10de10 con 0x7fb59c103ea0 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb591d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c102660 0x7fb59c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb591d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59054/0 (socket says 172.21.15.50:59054) 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb591d9b640 1 -- 172.21.15.50:0/111864955 learned_addr learned my addr 172.21.15.50:0/111864955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.938+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c103ea0 0x7fb59c1a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 -- 172.21.15.50:0/111864955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c103ea0 msgr2=0x7fb59c1a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c103ea0 0x7fb59c1a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 -- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c102660 msgr2=0x7fb59c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c102660 0x7fb59c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 -- 172.21.15.50:0/111864955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c1a1590 con 0x7fb59c103550 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb591d9b640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb584005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb593fff640 1 -- 172.21.15.50:0/111864955 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58403f030 con 0x7fb59c103550 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb593fff640 1 -- 172.21.15.50:0/111864955 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5840043d0 con 0x7fb59c103550 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59c1a1820 con 0x7fb59c103550 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb593fff640 1 -- 172.21.15.50:0/111864955 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584035070 con 0x7fb59c103550 2024-05-17T04:43:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.939+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59c1a1d10 con 0x7fb59c103550 2024-05-17T04:43:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.943+0000 7fb593fff640 1 -- 172.21.15.50:0/111864955 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb584042030 con 0x7fb59c103550 2024-05-17T04:43:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 Processor -- start 2024-05-17T04:43:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 -- start start 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714410dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x557144114c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55714407fcc0 con 0x557143d494d0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55714406f860 con 0x557143d412f0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.944+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571440738a0 con 0x557143d6a1b0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x557144114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714410dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:35138/0 (socket says 172.21.15.50:35138) 2024-05-17T04:43:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714410dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59056/0 (socket says 172.21.15.50:59056) 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59ad9d640 1 -- 172.21.15.50:0/1023540878 learned_addr learned my addr 172.21.15.50:0/1023540878 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb593fff640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 0x7fb5880b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 -- 172.21.15.50:0/1023540878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 msgr2=0x55714410d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 -- 172.21.15.50:0/1023540878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 msgr2=0x55714410dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714410dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 -- 172.21.15.50:0/1023540878 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557144037d50 con 0x557143d494d0 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 0x7fb5880b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb593fff640 1 -- 172.21.15.50:0/111864955 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb584102a20 con 0x7fb59c103550 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.945+0000 7fb591d9b640 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x557144114c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb584100160 tx=0x7fb5840944d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5125cabfcc8fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.946+0000 7fb590d99640 1 -- 172.21.15.50:0/1023540878 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5840c59a0 con 0x557143d494d0 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.946+0000 7fb590d99640 1 -- 172.21.15.50:0/1023540878 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5840c97c0 con 0x557143d494d0 2024-05-17T04:43:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.946+0000 7fb590d99640 1 -- 172.21.15.50:0/1023540878 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5840c9420 con 0x557143d494d0 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 -- 172.21.15.50:0/1023540878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 msgr2=0x557144114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 0x7fb5880b4cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55714410e960 tx=0x7fb594004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x557144114c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb584100160 tx=0x7fb5840944d0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 -- 172.21.15.50:0/1023540878 shutdown_connections 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x557144114c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/1023540878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 -- 172.21.15.50:0/1023540878 >> 172.21.15.50:0/1023540878 conn(0x557143d5e300 msgr2=0x5571441091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 -- 172.21.15.50:0/1023540878 shutdown_connections 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.947+0000 7fb5a77fc740 1 -- 172.21.15.50:0/1023540878 wait complete. 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 -- start start 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 Processor -- start 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x5571441171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557143fabab0 con 0x557143d6a1b0 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55714402c4d0 con 0x557143d412f0 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557144115450 con 0x557143d494d0 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb5a77fc740 10 client.0 _ll_register_callbacks cb 0x55714406a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb591d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x5571441171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59060/0 (socket says 172.21.15.50:59060) 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59a59c640 1 -- 172.21.15.50:0/837017308 learned_addr learned my addr 172.21.15.50:0/837017308 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59ad9d640 1 -- 172.21.15.50:0/837017308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 msgr2=0x5571441171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x5571441171e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59ad9d640 1 -- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 msgr2=0x557144116ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.948+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb59ad9d640 1 -- 172.21.15.50:0/837017308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55714402d340 con 0x557143d6a1b0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb59a59c640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411e140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb594009fd0 tx=0x7fb594004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bab8318a3dfd81 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59402e060 con 0x557143d6a1b0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb59401dd50 con 0x557143d6a1b0 2024-05-17T04:43:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5840c97c0 con 0x557143d6a1b0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb594035050 con 0x557143d6a1b0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb5a77fc740 10 client.15867 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.949+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557143dd5790 con 0x557143d6a1b0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5840c97c0 con 0x557143d6a1b0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb59400f3c0 con 0x557143d6a1b0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 10 client.15867 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 20 client.15867 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143d28d80 0x5571441244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 4 client.15867 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557144081c10 con 0x557143d28d80 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.950+0000 7fb59a59c640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143d28d80 0x5571441244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.951+0000 7fb581ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.951+0000 7fb59a59c640 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143d28d80 0x5571441244b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6693803123e41b server_cookie=1caf20a1e852b3e7 in_seq=0 out_seq=0 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.951+0000 7fb581ffb640 1 -- 172.21.15.50:0/837017308 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557144081c10 con 0x557143d28d80 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.951+0000 7fb581ffb640 10 client.15867 handle_command_reply: tid=0 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 client.15867 shutdown 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 msgr2=0x55714411e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411e140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb594009fd0 tx=0x7fb594004ab0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 shutdown_connections 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143d28d80 0x5571441244b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411e140 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x5571441171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb581ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/837017308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x557144116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 >> 172.21.15.50:0/837017308 conn(0x557143d5e300 msgr2=0x55714410c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 shutdown_connections 2024-05-17T04:43:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 1 -- 172.21.15.50:0/837017308 wait complete. 2024-05-17T04:43:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.952+0000 7fb5a77fc740 20 client.15867 trim_cache size 0 max 16384 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 Processor -- start 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 -- start start 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x55714411e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714411e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1ac6a0 con 0x557143d494d0 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1a9f70 con 0x557143d412f0 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1a7dc0 con 0x557143d6a1b0 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59064/0 (socket says 172.21.15.50:59064) 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.974+0000 7fb59a59c640 1 -- 172.21.15.50:0/2968897910 learned_addr learned my addr 172.21.15.50:0/2968897910 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714411e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb591d9b640 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x55714411e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 -- 172.21.15.50:0/2968897910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 msgr2=0x55714411e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714411e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 -- 172.21.15.50:0/2968897910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 msgr2=0x55714411e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x55714411e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 -- 172.21.15.50:0/2968897910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557143e4e620 con 0x557143d412f0 2024-05-17T04:43:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d417bfd97624817 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb583fff640 1 -- 172.21.15.50:0/2968897910 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c024020 con 0x557143d412f0 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb583fff640 1 -- 172.21.15.50:0/2968897910 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb58c020840 con 0x557143d412f0 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.975+0000 7fb583fff640 1 -- 172.21.15.50:0/2968897910 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c010b30 con 0x557143d412f0 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2968897910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 msgr2=0x5571441109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000c00 tx=0x7fb58c005f90 comp rx=0 tx=0).stop 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2968897910 shutdown_connections 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d6a1b0 0x55714411e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d494d0 0x55714411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2968897910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x5571441109a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2968897910 >> 172.21.15.50:0/2968897910 conn(0x557143d5e300 msgr2=0x55714410c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2968897910 shutdown_connections 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.977+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2968897910 wait complete. 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 -- start start 2024-05-17T04:43:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 Processor -- start 2024-05-17T04:43:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x55714411c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1a8430 con 0x557143d6a1b0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1a98a0 con 0x557143d412f0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb5a77fc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1a8b00 con 0x557143d494d0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.978+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb5a77fc740 10 client.0 _ll_register_callbacks cb 0x7fb594003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:59072/0 (socket says 172.21.15.50:59072) 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 -- 172.21.15.50:0/2778108423 learned_addr learned my addr 172.21.15.50:0/2778108423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb591d9b640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x55714411c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 -- 172.21.15.50:0/2778108423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 msgr2=0x55714411c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x55714411c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 -- 172.21.15.50:0/2778108423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 msgr2=0x55714411cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 -- 172.21.15.50:0/2778108423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55714411d550 con 0x557143d412f0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59ad9d640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb59a59c640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00a010 tx=0x7fb58c020ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ead0cef6ba4acd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.979+0000 7fb581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c024030 con 0x557143d412f0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb58c0203b0 con 0x557143d412f0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58c020840 con 0x557143d412f0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb58c03a050 con 0x557143d412f0 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb5a77fc740 10 client.25631 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55714411e810 con 0x557143d412f0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb58c02e070 con 0x557143d412f0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.980+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb58c0053d0 con 0x557143d412f0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 10 client.25631 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 20 client.25631 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143cfe1c0 0x55714418ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 4 client.25631 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb58c020840 con 0x557143cfe1c0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb591d9b640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143cfe1c0 0x55714418ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb591d9b640 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143cfe1c0 0x55714418ebb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e318ccae55ebef60 server_cookie=9435d95065c76657 in_seq=0 out_seq=0 2024-05-17T04:43:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:35.981+0000 7fb581ffb640 10 client.25631 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb581ffb640 1 -- 172.21.15.50:0/2778108423 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb58c020840 con 0x557143cfe1c0 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb581ffb640 10 client.25631 handle_command_reply: tid=0 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 client.25631 shutdown 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 msgr2=0x55714411bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00a010 tx=0x7fb58c020ae0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 shutdown_connections 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557143cfe1c0 0x55714418ebb0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557143d6a1b0 0x55714411cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557143d494d0 0x55714411c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb581ffb640 0 client.25631 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.009+0000 7fb5a77fc740 1 --2- 172.21.15.50:0/2778108423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557143d412f0 0x55714411bb60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 >> 172.21.15.50:0/2778108423 conn(0x557143d5e300 msgr2=0x5571441077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 shutdown_connections 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a77fc740 1 -- 172.21.15.50:0/2778108423 wait complete. 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a77fc740 20 client.25631 trim_cache size 0 max 16384 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:43:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:43:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 msgr2=0x7fb5880b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 0x7fb5880b4cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55714410e960 tx=0x7fb594004a40 comp rx=0 tx=0).stop 2024-05-17T04:43:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 msgr2=0x7fb59c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb584005e80 comp rx=0 tx=0).stop 2024-05-17T04:43:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 shutdown_connections 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb59c103ea0 0x7fb59c1a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb59c103550 0x7fb59c199f90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb5880b27e0 0x7fb5880b4cf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 --2- 172.21.15.50:0/111864955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb59c102660 0x7fb59c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.010+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 >> 172.21.15.50:0/111864955 conn(0x7fb59c0fd010 msgr2=0x7fb59c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.011+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 shutdown_connections 2024-05-17T04:43:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:36.011+0000 7fb5a0865640 1 -- 172.21.15.50:0/111864955 wait complete. 2024-05-17T04:43:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:36 smithi105 ceph-mon[34795]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-05-17T04:43:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:36 smithi110 ceph-mon[35497]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-05-17T04:43:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:36 smithi050 ceph-mon[31739]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.5 MiB/s wr, 347 op/s 2024-05-17T04:43:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 280.0 s, 145.2 tps, lat 756.910 ms stddev 229.447 2024-05-17T04:43:38.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:38 smithi105 ceph-mon[34795]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-05-17T04:43:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:38 smithi110 ceph-mon[35497]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-05-17T04:43:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:38 smithi050 ceph-mon[31739]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 3.5 MiB/s wr, 299 op/s 2024-05-17T04:43:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:40 smithi105 ceph-mon[34795]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-17T04:43:40.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:40 smithi110 ceph-mon[35497]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-17T04:43:40.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:40 smithi050 ceph-mon[31739]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-05-17T04:43:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:42 smithi105 ceph-mon[34795]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.3 MiB/s wr, 324 op/s 2024-05-17T04:43:42.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:42 smithi110 ceph-mon[35497]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.3 MiB/s wr, 324 op/s 2024-05-17T04:43:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:42 smithi050 ceph-mon[31739]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.3 MiB/s wr, 324 op/s 2024-05-17T04:43:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 285.0 s, 107.2 tps, lat 926.093 ms stddev 394.037 2024-05-17T04:43:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:44 smithi105 ceph-mon[34795]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:43:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:44 smithi110 ceph-mon[35497]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:43:44.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:44 smithi050 ceph-mon[31739]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.2 MiB/s wr, 405 op/s 2024-05-17T04:43:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:46 smithi105 ceph-mon[34795]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.9 MiB/s wr, 438 op/s 2024-05-17T04:43:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:46 smithi110 ceph-mon[35497]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.9 MiB/s wr, 438 op/s 2024-05-17T04:43:46.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:46 smithi050 ceph-mon[31739]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.9 MiB/s wr, 438 op/s 2024-05-17T04:43:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 290.0 s, 143.6 tps, lat 684.788 ms stddev 224.780 2024-05-17T04:43:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:48 smithi105 ceph-mon[34795]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-05-17T04:43:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:48 smithi110 ceph-mon[35497]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-05-17T04:43:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:48 smithi050 ceph-mon[31739]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 -- 172.21.15.50:0/3270575063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c103c80 msgr2=0x7fd88c1060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 --2- 172.21.15.50:0/3270575063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c103c80 0x7fd88c1060a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88802a3f0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 -- 172.21.15.50:0/3270575063 shutdown_connections 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 --2- 172.21.15.50:0/3270575063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c108f40 0x7fd88c10b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 --2- 172.21.15.50:0/3270575063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c1065e0 0x7fd88c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 --2- 172.21.15.50:0/3270575063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c103c80 0x7fd88c1060a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 -- 172.21.15.50:0/3270575063 >> 172.21.15.50:0/3270575063 conn(0x7fd88c0fd8b0 msgr2=0x7fd88c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 -- 172.21.15.50:0/3270575063 shutdown_connections 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.148+0000 7fd893e47640 1 -- 172.21.15.50:0/3270575063 wait complete. 2024-05-17T04:43:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 Processor -- start 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 -- start start 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c108f40 0x7fd88c1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c110da0 con 0x7fd88c1065e0 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c110c20 con 0x7fd88c108f40 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd893e47640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c110f20 con 0x7fd88c103c80 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd8913e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd891be3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd8913e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33854/0 (socket says 172.21.15.50:33854) 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd8913e2640 1 -- 172.21.15.50:0/3618730870 learned_addr learned my addr 172.21.15.50:0/3618730870 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd891be3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57486/0 (socket says 172.21.15.50:57486) 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.149+0000 7fd8923e4640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c108f40 0x7fd88c1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 -- 172.21.15.50:0/3618730870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c108f40 msgr2=0x7fd88c1a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c108f40 0x7fd88c1a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 -- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 msgr2=0x7fd88c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 -- 172.21.15.50:0/3618730870 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c1a3510 con 0x7fd88c103c80 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd8913e2640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd891be3640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd888009230 tx=0x7fd888005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd882ffd640 1 -- 172.21.15.50:0/3618730870 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88803f070 con 0x7fd88c103c80 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd882ffd640 1 -- 172.21.15.50:0/3618730870 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd888003a80 con 0x7fd88c103c80 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a37a0 con 0x7fd88c103c80 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd882ffd640 1 -- 172.21.15.50:0/3618730870 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x7fd88c103c80 2024-05-17T04:43:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.150+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a3c60 con 0x7fd88c103c80 2024-05-17T04:43:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.154+0000 7fd89ade9740 1 Processor -- start 2024-05-17T04:43:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.154+0000 7fd89ade9740 1 -- start start 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca19cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd882ffd640 1 -- 172.21.15.50:0/3618730870 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd888043400 con 0x7fd88c103c80 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5ac98bcc0 con 0x55c5ac64d2f0 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c5ac97b860 con 0x55c5ac6554d0 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd89ade9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5ac97f8a0 con 0x55c5ac6761b0 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd891be3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca19cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd891be3640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca19cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33862/0 (socket says 172.21.15.50:33862) 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57490/0 (socket says 172.21.15.50:57490) 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd891be3640 1 -- 172.21.15.50:0/1186610294 learned_addr learned my addr 172.21.15.50:0/1186610294 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd8913e2640 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd8923e4640 1 -- 172.21.15.50:0/1186610294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 msgr2=0x55c5aca20c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.155+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca20c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd8923e4640 1 -- 172.21.15.50:0/1186610294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 msgr2=0x55c5aca19cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca19cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd8923e4640 1 -- 172.21.15.50:0/1186610294 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ac943d50 con 0x55c5ac6761b0 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd884009f00 tx=0x7fd884011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=212d71e0d86971b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd882ffd640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 0x7fd8780b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd877fff640 1 -- 172.21.15.50:0/1186610294 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884004d20 con 0x55c5ac6761b0 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd877fff640 1 -- 172.21.15.50:0/1186610294 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd884004030 con 0x55c5ac6761b0 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd882ffd640 1 -- 172.21.15.50:0/3618730870 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd8881026e0 con 0x7fd88c103c80 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd891be3640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 0x7fd8780b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.156+0000 7fd877fff640 1 -- 172.21.15.50:0/1186610294 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884005260 con 0x55c5ac6761b0 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 -- 172.21.15.50:0/1186610294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 msgr2=0x55c5aca19770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd891be3640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 0x7fd8780b4cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880b1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd884009f00 tx=0x7fd884011810 comp rx=0 tx=0).stop 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 -- 172.21.15.50:0/1186610294 shutdown_connections 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca20c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca19cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1186610294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca19770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 -- 172.21.15.50:0/1186610294 >> 172.21.15.50:0/1186610294 conn(0x55c5ac66a300 msgr2=0x55c5aca151a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 -- 172.21.15.50:0/1186610294 shutdown_connections 2024-05-17T04:43:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.157+0000 7fd89ade9740 1 -- 172.21.15.50:0/1186610294 wait complete. 2024-05-17T04:43:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 -- start start 2024-05-17T04:43:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 Processor -- start 2024-05-17T04:43:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5ac8b7ab0 con 0x55c5ac64d2f0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c5ac9384d0 con 0x55c5ac6761b0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5aca21690 con 0x55c5ac6554d0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.158+0000 7fd89ade9740 10 client.0 _ll_register_callbacks cb 0x55c5ac976950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33878/0 (socket says 172.21.15.50:33878) 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 -- 172.21.15.50:0/2686495905 learned_addr learned my addr 172.21.15.50:0/2686495905 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8913e2640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 -- 172.21.15.50:0/2686495905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 msgr2=0x55c5aca23570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca23570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 -- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6761b0 msgr2=0x55c5aca2a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 -- 172.21.15.50:0/2686495905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ac939340 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8923e4640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd8840049d0 tx=0x7fd884000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e33c2d5d7ab8d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8757fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88402e060 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd88400f890 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd884004030 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd88401e040 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.159+0000 7fd89ade9740 10 client.15885 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c5ac6e1790 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd884004030 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd884018070 con 0x55c5ac64d2f0 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 10 client.15885 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 20 client.15885 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac634d80 0x55c5aca305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 4 client.15885 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ac98dc10 con 0x55c5ac634d80 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.160+0000 7fd8913e2640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac634d80 0x55c5aca305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.161+0000 7fd8757fa640 10 client.15885 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.161+0000 7fd8913e2640 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac634d80 0x55c5aca305c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cfbca7f997e242 server_cookie=39bd95343427bd34 in_seq=0 out_seq=0 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.161+0000 7fd8757fa640 1 -- 172.21.15.50:0/2686495905 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5ac98dc10 con 0x55c5ac634d80 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.161+0000 7fd8757fa640 10 client.15885 handle_command_reply: tid=0 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 client.15885 shutdown 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 msgr2=0x55c5aca23030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd8840049d0 tx=0x7fd884000f40 comp rx=0 tx=0).stop 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 shutdown_connections 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca23570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac634d80 0x55c5aca305c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 --2- 172.21.15.50:0/2686495905 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca23030 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 >> 172.21.15.50:0/2686495905 conn(0x55c5ac66a300 msgr2=0x55c5aca141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd8757fa640 0 client.15885 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 shutdown_connections 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 1 -- 172.21.15.50:0/2686495905 wait complete. 2024-05-17T04:43:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.162+0000 7fd89ade9740 20 client.15885 trim_cache size 0 max 16384 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 Processor -- start 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 -- start start 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac64d2f0 0x55c5aca13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca2a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1b0a80 con 0x55c5ac6761b0 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1ae350 con 0x55c5ac64d2f0 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd89ade9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1ac1a0 con 0x55c5ac6554d0 2024-05-17T04:43:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33894/0 (socket says 172.21.15.50:33894) 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd8923e4640 1 -- 172.21.15.50:0/3878137532 learned_addr learned my addr 172.21.15.50:0/3878137532 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd8913e2640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca2a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.183+0000 7fd8923e4640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac64d2f0 0x55c5aca13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 -- 172.21.15.50:0/3878137532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac64d2f0 msgr2=0x55c5aca13ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac64d2f0 0x55c5aca13ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 -- 172.21.15.50:0/3878137532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 msgr2=0x55c5aca2a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 -- 172.21.15.50:0/3878137532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ac75a620 con 0x55c5ac6554d0 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8923e4640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8913e2640 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca2a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55c5ac8b7c00 tx=0x7fd87c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60625b9bb150b88f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.184+0000 7fd8777fe640 1 -- 172.21.15.50:0/3878137532 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c024070 con 0x55c5ac6554d0 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.185+0000 7fd8777fe640 1 -- 172.21.15.50:0/3878137532 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd87c0054b0 con 0x55c5ac6554d0 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.185+0000 7fd8777fe640 1 -- 172.21.15.50:0/3878137532 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c010d20 con 0x55c5ac6554d0 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- 172.21.15.50:0/3878137532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 msgr2=0x55c5aca2a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca2a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55c5ac8b7c00 tx=0x7fd87c005f90 comp rx=0 tx=0).stop 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- 172.21.15.50:0/3878137532 shutdown_connections 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac6761b0 0x55c5aca2a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6554d0 0x55c5aca2a460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 --2- 172.21.15.50:0/3878137532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac64d2f0 0x55c5aca13ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- 172.21.15.50:0/3878137532 >> 172.21.15.50:0/3878137532 conn(0x55c5ac66a300 msgr2=0x55c5aca137e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- 172.21.15.50:0/3878137532 shutdown_connections 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- 172.21.15.50:0/3878137532 wait complete. 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.186+0000 7fd89ade9740 1 -- start start 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 Processor -- start 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca27f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1ac810 con 0x55c5ac64d2f0 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1adc80 con 0x55c5ac6554d0 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c1acee0 con 0x55c5ac6761b0 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd89ade9740 10 client.0 _ll_register_callbacks cb 0x7fd888108420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:43:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd8923e4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33904/0 (socket says 172.21.15.50:33904) 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd8923e4640 1 -- 172.21.15.50:0/1728802668 learned_addr learned my addr 172.21.15.50:0/1728802668 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd8913e2640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca27f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.187+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 -- 172.21.15.50:0/1728802668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 msgr2=0x55c5aca96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca96920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 -- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 msgr2=0x55c5aca27f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca27f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 -- 172.21.15.50:0/1728802668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5aca2af40 con 0x55c5ac64d2f0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8913e2640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8923e4640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd884009d40 tx=0x7fd884004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb3044a50f41c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8757fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88402e070 con 0x55c5ac64d2f0 2024-05-17T04:43:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd884028400 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.188+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87c0054b0 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd884049050 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 10 client.15891 fetch_fsmap learned FSMap version 39 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c5aca97e20 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd87c0054b0 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd884018070 con 0x55c5ac64d2f0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 10 client.15891 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 20 client.15891 populate_metadata read hostname 'smithi050' 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac60a1c0 0x55c5aca9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 4 client.15891 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.189+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5ac9873f0 con 0x55c5ac60a1c0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.190+0000 7fd8913e2640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac60a1c0 0x55c5aca9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:43:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.190+0000 7fd8913e2640 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac60a1c0 0x55c5aca9a750 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a09401cba7b6593a server_cookie=f6d2f73e4d8442e2 in_seq=0 out_seq=0 2024-05-17T04:43:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.190+0000 7fd8757fa640 10 client.15891 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.208+0000 7fd8757fa640 1 -- 172.21.15.50:0/1728802668 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c5ac9873f0 con 0x55c5ac60a1c0 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.208+0000 7fd8757fa640 10 client.15891 handle_command_reply: tid=0 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.208+0000 7fd89ade9740 1 client.15891 shutdown 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.208+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 msgr2=0x55c5aca27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.208+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd884009d40 tx=0x7fd884004380 comp rx=0 tx=0).stop 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 shutdown_connections 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5ac6761b0 0x55c5aca96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5ac6554d0 0x55c5aca27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5ac60a1c0 0x55c5aca9a750 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 --2- 172.21.15.50:0/1728802668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5ac64d2f0 0x55c5aca27a10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd8757fa640 0 client.15891 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 >> 172.21.15.50:0/1728802668 conn(0x55c5ac66a300 msgr2=0x55c5aca141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 shutdown_connections 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 1 -- 172.21.15.50:0/1728802668 wait complete. 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd89ade9740 20 client.15891 trim_cache size 0 max 16384 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:43:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 msgr2=0x7fd8780b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 0x7fd8780b4cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880b1040 comp rx=0 tx=0).stop 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 msgr2=0x7fd88c19b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd888009230 tx=0x7fd888005e20 comp rx=0 tx=0).stop 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 shutdown_connections 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd88c108f40 0x7fd88c1a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd8780b27e0 0x7fd8780b4cf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd88c1065e0 0x7fd88c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 --2- 172.21.15.50:0/3618730870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c103c80 0x7fd88c19b9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:43:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.209+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 >> 172.21.15.50:0/3618730870 conn(0x7fd88c0fd8b0 msgr2=0x7fd88c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:43:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.210+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 shutdown_connections 2024-05-17T04:43:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:43:49.210+0000 7fd893e47640 1 -- 172.21.15.50:0/3618730870 wait complete. 2024-05-17T04:43:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:50 smithi105 ceph-mon[34795]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.5 MiB/s wr, 500 op/s 2024-05-17T04:43:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:50 smithi110 ceph-mon[35497]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.5 MiB/s wr, 500 op/s 2024-05-17T04:43:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:50 smithi050 ceph-mon[31739]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.5 MiB/s wr, 500 op/s 2024-05-17T04:43:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:52 smithi105 ceph-mon[34795]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-05-17T04:43:52.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:52 smithi110 ceph-mon[35497]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-05-17T04:43:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:52 smithi050 ceph-mon[31739]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-05-17T04:43:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 295.0 s, 132.4 tps, lat 766.567 ms stddev 183.749 2024-05-17T04:43:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:54 smithi105 ceph-mon[34795]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-17T04:43:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:54 smithi110 ceph-mon[35497]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-17T04:43:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:54 smithi050 ceph-mon[31739]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-17T04:43:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:56 smithi105 ceph-mon[34795]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-05-17T04:43:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:56 smithi110 ceph-mon[35497]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-05-17T04:43:56.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:56 smithi050 ceph-mon[31739]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-05-17T04:43:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 300.0 s, 121.6 tps, lat 805.004 ms stddev 185.336 2024-05-17T04:43:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:43:58 smithi105 ceph-mon[34795]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-05-17T04:43:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:43:58 smithi110 ceph-mon[35497]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-05-17T04:43:58.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:43:58 smithi050 ceph-mon[31739]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-05-17T04:44:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:00 smithi105 ceph-mon[34795]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-17T04:44:00.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:00 smithi110 ceph-mon[35497]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-17T04:44:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:00 smithi050 ceph-mon[31739]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 -- 172.21.15.50:0/2339202238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c108e30 msgr2=0x7f4e3c10b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 --2- 172.21.15.50:0/2339202238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c108e30 0x7f4e3c10b340 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009ca0 tx=0x7f4e3402a8c0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 -- 172.21.15.50:0/2339202238 shutdown_connections 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 --2- 172.21.15.50:0/2339202238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c108e30 0x7f4e3c10b340 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 --2- 172.21.15.50:0/2339202238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 0x7f4e3c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.329+0000 7f4e443cb640 1 --2- 172.21.15.50:0/2339202238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.330+0000 7f4e443cb640 1 -- 172.21.15.50:0/2339202238 >> 172.21.15.50:0/2339202238 conn(0x7f4e3c0fd880 msgr2=0x7f4e3c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.330+0000 7f4e443cb640 1 -- 172.21.15.50:0/2339202238 shutdown_connections 2024-05-17T04:44:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.330+0000 7f4e443cb640 1 -- 172.21.15.50:0/2339202238 wait complete. 2024-05-17T04:44:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.330+0000 7f4e443cb640 1 Processor -- start 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.330+0000 7f4e443cb640 1 -- start start 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 0x7f4e3c101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c108e30 0x7f4e3c102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10ebb0 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10ea30 con 0x7f4e3c108e30 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e443cb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10ed30 con 0x7f4e3c1062f0 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49018/0 (socket says 172.21.15.50:49018) 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 -- 172.21.15.50:0/1457046662 learned_addr learned my addr 172.21.15.50:0/1457046662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e41966640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 0x7f4e3c101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42968640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c108e30 0x7f4e3c102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 -- 172.21.15.50:0/1457046662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 msgr2=0x7f4e3c101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 0x7f4e3c101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 -- 172.21.15.50:0/1457046662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c108e30 msgr2=0x7f4e3c102160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c108e30 0x7f4e3c102160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 -- 172.21.15.50:0/1457046662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c102890 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.331+0000 7f4e42167640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e9c0 tx=0x7f4e3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.332+0000 7f4e337fe640 1 -- 172.21.15.50:0/1457046662 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800cde0 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.332+0000 7f4e337fe640 1 -- 172.21.15.50:0/1457046662 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e380108c0 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.332+0000 7f4e337fe640 1 -- 172.21.15.50:0/1457046662 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38018870 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.332+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c1a3860 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.332+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1a3cd0 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.335+0000 7f4e4b282740 1 Processor -- start 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.335+0000 7f4e337fe640 1 -- 172.21.15.50:0/1457046662 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4e3801f020 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.335+0000 7f4e4b282740 1 -- start start 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.335+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d71f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d72500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 0x55ec49d79420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e4b282740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ec49cdbcc0 con 0x55ec499c61b0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e4b282740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ec49ccb860 con 0x55ec4999d2f0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e4b282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ec49ccf8a0 con 0x55ec499a54d0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e42968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d71f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e42167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d72500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e42968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d71f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49034/0 (socket says 172.21.15.50:49034) 2024-05-17T04:44:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e42968640 1 -- 172.21.15.50:0/1313042227 learned_addr learned my addr 172.21.15.50:0/1313042227 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 0x55ec49d79420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 -- 172.21.15.50:0/1313042227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 msgr2=0x55ec49d72500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d72500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 -- 172.21.15.50:0/1313042227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 msgr2=0x55ec49d71f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d71f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 -- 172.21.15.50:0/1313042227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec49c93d50 con 0x55ec499a54d0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.336+0000 7f4e41966640 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 0x55ec49d79420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c06b640 tx=0x7f4e2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5faec82598f2b5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.337+0000 7f4e30bfe640 1 -- 172.21.15.50:0/1313042227 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c01e070 con 0x55ec499a54d0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.337+0000 7f4e30bfe640 1 -- 172.21.15.50:0/1313042227 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c0035f0 con 0x55ec499a54d0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.337+0000 7f4e30bfe640 1 -- 172.21.15.50:0/1313042227 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0044e0 con 0x55ec499a54d0 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.337+0000 7f4e337fe640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 0x7f4e280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.338+0000 7f4e337fe640 1 -- 172.21.15.50:0/1457046662 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4e380e2720 con 0x7f4e3c06b1f0 2024-05-17T04:44:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.338+0000 7f4e42968640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 0x7f4e280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.338+0000 7f4e4b282740 1 -- 172.21.15.50:0/1313042227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 msgr2=0x55ec49d79420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.338+0000 7f4e4b282740 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 0x55ec49d79420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c06b640 tx=0x7f4e2c011810 comp rx=0 tx=0).stop 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 -- 172.21.15.50:0/1313042227 shutdown_connections 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499a54d0 0x55ec49d79420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d72500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 --2- 172.21.15.50:0/1313042227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d71f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 -- 172.21.15.50:0/1313042227 >> 172.21.15.50:0/1313042227 conn(0x55ec499ba300 msgr2=0x55ec49d6d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e42968640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 0x7f4e280b4cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094e0 tx=0x7f4e34031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 -- 172.21.15.50:0/1313042227 shutdown_connections 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.339+0000 7f4e4b282740 1 -- 172.21.15.50:0/1313042227 wait complete. 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.340+0000 7f4e4b282740 1 -- start start 2024-05-17T04:44:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.340+0000 7f4e4b282740 1 Processor -- start 2024-05-17T04:44:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.340+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.340+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:48640/0 (socket says 172.21.15.50:48640) 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d82d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 -- 172.21.15.50:0/879490008 learned_addr learned my addr 172.21.15.50:0/879490008 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ec49c07ab0 con 0x55ec499a54d0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ec49c884d0 con 0x55ec4999d2f0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ec49d79ea0 con 0x55ec499c61b0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e4b282740 10 client.0 _ll_register_callbacks cb 0x55ec49cc6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e41966640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 -- 172.21.15.50:0/879490008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 msgr2=0x55ec49d82d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d82d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 -- 172.21.15.50:0/879490008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 msgr2=0x55ec49d7bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.341+0000 7f4e42167640 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec49c89340 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e42167640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e380a96b0 tx=0x7f4e380a9840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2cf9f158e6f5508 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380a94c0 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4e380e2450 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e2c0035f0 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4e380c3050 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e4b282740 10 client.25661 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec49a31790 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c0035f0 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4e380aace0 con 0x55ec4999d2f0 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e4b282740 10 client.25661 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.342+0000 7f4e4b282740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e4b282740 20 client.25661 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec49984d80 0x55ec49d890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e4b282740 4 client.25661 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:44:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec49cddc10 con 0x55ec49984d80 2024-05-17T04:44:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e41966640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec49984d80 0x55ec49d890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e25ffb640 10 client.25661 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.343+0000 7f4e41966640 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec49984d80 0x55ec49d890e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce1cabda407aa74 server_cookie=4d671e81a46c6bed in_seq=0 out_seq=0 2024-05-17T04:44:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e25ffb640 1 -- 172.21.15.50:0/879490008 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec49cddc10 con 0x55ec49984d80 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e25ffb640 10 client.25661 handle_command_reply: tid=0 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 client.25661 shutdown 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 msgr2=0x55ec49d7b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e380a96b0 tx=0x7f4e380a9840 comp rx=0 tx=0).stop 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 shutdown_connections 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d82d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec49984d80 0x55ec49d890e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 --2- 172.21.15.50:0/879490008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d7b8a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 >> 172.21.15.50:0/879490008 conn(0x55ec499ba300 msgr2=0x55ec49d70820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.344+0000 7f4e25ffb640 0 client.25661 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.345+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 shutdown_connections 2024-05-17T04:44:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.345+0000 7f4e4b282740 1 -- 172.21.15.50:0/879490008 wait complete. 2024-05-17T04:44:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.345+0000 7f4e4b282740 20 client.25661 trim_cache size 0 max 16384 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 Processor -- start 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 -- start start 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 0x55ec49d6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 0x55ec49d82c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1ac750 con 0x55ec499c61b0 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1aa020 con 0x55ec499a54d0 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e4b282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1a7e70 con 0x55ec4999d2f0 2024-05-17T04:44:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e42968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e41966640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 0x55ec49d82c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e42968640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49058/0 (socket says 172.21.15.50:49058) 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.365+0000 7f4e42167640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 0x55ec49d6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e42968640 1 -- 172.21.15.50:0/3377809456 learned_addr learned my addr 172.21.15.50:0/3377809456 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 -- 172.21.15.50:0/3377809456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 msgr2=0x55ec49d6da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 0x55ec49d6da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 -- 172.21.15.50:0/3377809456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 msgr2=0x55ec49d706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 -- 172.21.15.50:0/3377809456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec49aaa620 con 0x55ec499a54d0 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e42167640 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 0x55ec49d6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e41966640 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 0x55ec49d82c60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c005e00 tx=0x7f4e2c0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7104d2e52351e15e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e27fff640 1 -- 172.21.15.50:0/3377809456 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c01e040 con 0x55ec499a54d0 2024-05-17T04:44:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e27fff640 1 -- 172.21.15.50:0/3377809456 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c01f040 con 0x55ec499a54d0 2024-05-17T04:44:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.366+0000 7f4e27fff640 1 -- 172.21.15.50:0/3377809456 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c02c4c0 con 0x55ec499a54d0 2024-05-17T04:44:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 -- 172.21.15.50:0/3377809456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 msgr2=0x55ec49d82c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 0x55ec49d82c60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c005e00 tx=0x7f4e2c0096c0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 -- 172.21.15.50:0/3377809456 shutdown_connections 2024-05-17T04:44:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499c61b0 0x55ec49d706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec499a54d0 0x55ec49d82c60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 --2- 172.21.15.50:0/3377809456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec4999d2f0 0x55ec49d6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 -- 172.21.15.50:0/3377809456 >> 172.21.15.50:0/3377809456 conn(0x55ec499ba300 msgr2=0x55ec49d6d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 -- 172.21.15.50:0/3377809456 shutdown_connections 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.367+0000 7f4e4b282740 1 -- 172.21.15.50:0/3377809456 wait complete. 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 -- start start 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 Processor -- start 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d7fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1a84e0 con 0x55ec499a54d0 2024-05-17T04:44:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1a9950 con 0x55ec4999d2f0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1a8bb0 con 0x55ec499c61b0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e4b282740 10 client.0 _ll_register_callbacks cb 0x7f4e380a5870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e41966640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e41966640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49064/0 (socket says 172.21.15.50:49064) 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e41966640 1 -- 172.21.15.50:0/2153177416 learned_addr learned my addr 172.21.15.50:0/2153177416 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e42167640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.368+0000 7f4e42968640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d7fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 -- 172.21.15.50:0/2153177416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 msgr2=0x55ec49d7fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d7fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 -- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 msgr2=0x55ec49d82b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d82b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 -- 172.21.15.50:0/2153177416 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec49def460 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e42167640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e41966640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0099b0 tx=0x7f4e2c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38a035f25a1ef86 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c039070 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4e2c005050 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e2c01f040 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4e2c045050 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e4b282740 10 client.15909 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec49df05f0 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.369+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c01f040 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4e2c034070 con 0x55ec499a54d0 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 10 client.15909 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 20 client.15909 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec4995a1c0 0x55ec49df2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 4 client.15909 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec49cd73f0 con 0x55ec4995a1c0 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e42167640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec4995a1c0 0x55ec49df2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e25ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.370+0000 7f4e42167640 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec4995a1c0 0x55ec49df2db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3b129059940f53 server_cookie=3395d7ed96cfab5d in_seq=0 out_seq=0 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e25ffb640 1 -- 172.21.15.50:0/2153177416 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ec49cd73f0 con 0x55ec4995a1c0 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e25ffb640 10 client.15909 handle_command_reply: tid=0 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 client.15909 shutdown 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 msgr2=0x55ec49d7f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0099b0 tx=0x7f4e2c009e90 comp rx=0 tx=0).stop 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 shutdown_connections 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec499c61b0 0x55ec49d7fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ec4995a1c0 0x55ec49df2db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ec499a54d0 0x55ec49d7f6e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 --2- 172.21.15.50:0/2153177416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ec4999d2f0 0x55ec49d82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e25ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 >> 172.21.15.50:0/2153177416 conn(0x55ec499ba300 msgr2=0x55ec49d637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 shutdown_connections 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 1 -- 172.21.15.50:0/2153177416 wait complete. 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.387+0000 7f4e4b282740 20 client.15909 trim_cache size 0 max 16384 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:44:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 msgr2=0x7f4e280b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 0x7f4e280b4cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094e0 tx=0x7f4e34031000 comp rx=0 tx=0).stop 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 msgr2=0x7f4e3c103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e9c0 tx=0x7f4e3800ee80 comp rx=0 tx=0).stop 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 shutdown_connections 2024-05-17T04:44:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e3c108e30 0x7f4e3c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c1062f0 0x7f4e3c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4e280b27e0 0x7f4e280b4cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 --2- 172.21.15.50:0/1457046662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e3c06b1f0 0x7f4e3c103580 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 >> 172.21.15.50:0/1457046662 conn(0x7f4e3c0fd880 msgr2=0x7f4e3c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 shutdown_connections 2024-05-17T04:44:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:02.388+0000 7f4e443cb640 1 -- 172.21.15.50:0/1457046662 wait complete. 2024-05-17T04:44:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:02 smithi105 ceph-mon[34795]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-05-17T04:44:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:02 smithi110 ceph-mon[35497]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-05-17T04:44:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:02 smithi050 ceph-mon[31739]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-05-17T04:44:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 305.0 s, 132.8 tps, lat 774.442 ms stddev 203.084 2024-05-17T04:44:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:04 smithi105 ceph-mon[34795]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-05-17T04:44:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:04 smithi110 ceph-mon[35497]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-05-17T04:44:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:04 smithi050 ceph-mon[31739]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-05-17T04:44:05.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:05 smithi105 ceph-mon[34795]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:05.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:05 smithi110 ceph-mon[35497]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:05.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:05 smithi050 ceph-mon[31739]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 310.0 s, 132.8 tps, lat 511.029 ms stddev 260.025 2024-05-17T04:44:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:07 smithi105 ceph-mon[34795]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-05-17T04:44:08.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:07 smithi110 ceph-mon[35497]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-05-17T04:44:08.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:07 smithi050 ceph-mon[31739]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 282 op/s 2024-05-17T04:44:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:09 smithi105 ceph-mon[34795]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-05-17T04:44:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:09 smithi110 ceph-mon[35497]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-05-17T04:44:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:09 smithi050 ceph-mon[31739]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-05-17T04:44:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:11 smithi105 ceph-mon[34795]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:11 smithi110 ceph-mon[35497]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:12.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:11 smithi050 ceph-mon[31739]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-05-17T04:44:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 315.0 s, 121.0 tps, lat 301.320 ms stddev 70.368 2024-05-17T04:44:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:13 smithi105 ceph-mon[34795]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-17T04:44:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:13 smithi110 ceph-mon[35497]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-17T04:44:14.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:13 smithi050 ceph-mon[31739]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 -- 172.21.15.50:0/728131280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 msgr2=0x7f914010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 --2- 172.21.15.50:0/728131280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f914010a470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9138009ca0 tx=0x7f913802a8c0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 -- 172.21.15.50:0/728131280 shutdown_connections 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 --2- 172.21.15.50:0/728131280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f914010a470 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 --2- 172.21.15.50:0/728131280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f9140103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 --2- 172.21.15.50:0/728131280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f9140102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 -- 172.21.15.50:0/728131280 >> 172.21.15.50:0/728131280 conn(0x7f91400fd900 msgr2=0x7f91400ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 -- 172.21.15.50:0/728131280 shutdown_connections 2024-05-17T04:44:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.515+0000 7f914769e640 1 -- 172.21.15.50:0/728131280 wait complete. 2024-05-17T04:44:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.516+0000 7f914769e640 1 Processor -- start 2024-05-17T04:44:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.516+0000 7f914769e640 1 -- start start 2024-05-17T04:44:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.516+0000 7f914769e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f914006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.516+0000 7f914769e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f914019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.516+0000 7f914769e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914769e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f914010dac0 con 0x7f9140103b70 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914769e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f914010d940 con 0x7f9140102100 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914769e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f914010dc40 con 0x7f9140103220 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f914006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9144c39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f914019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f914006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38742/0 (socket says 172.21.15.50:38742) 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9144c39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f914019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58922/0 (socket says 172.21.15.50:58922) 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55638/0 (socket says 172.21.15.50:55638) 2024-05-17T04:44:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f914543a640 1 -- 172.21.15.50:0/2276859540 learned_addr learned my addr 172.21.15.50:0/2276859540 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 -- 172.21.15.50:0/2276859540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 msgr2=0x7f914019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f914019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 -- 172.21.15.50:0/2276859540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 msgr2=0x7f914006bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f914006bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 -- 172.21.15.50:0/2276859540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401a1240 con 0x7f9140103b70 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f9145c3b640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f913802ade0 tx=0x7f91380093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.517+0000 7f91367fc640 1 -- 172.21.15.50:0/2276859540 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138003cf0 con 0x7f9140103b70 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.518+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91401a14d0 con 0x7f9140103b70 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.518+0000 7f91367fc640 1 -- 172.21.15.50:0/2276859540 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91380044d0 con 0x7f9140103b70 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.518+0000 7f91367fc640 1 -- 172.21.15.50:0/2276859540 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138030b00 con 0x7f9140103b70 2024-05-17T04:44:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.518+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a1a70 con 0x7f9140103b70 2024-05-17T04:44:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.521+0000 7f914e638740 1 Processor -- start 2024-05-17T04:44:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f91367fc640 1 -- 172.21.15.50:0/2276859540 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9138030cc0 con 0x7f9140103b70 2024-05-17T04:44:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 -- start start 2024-05-17T04:44:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 0x564aae6a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2df4d0 0x564aae6aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564aae615cc0 con 0x564aae2df4d0 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564aae605860 con 0x564aae2d72f0 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914e638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564aae6098a0 con 0x564aae3001b0 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f9144c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2df4d0 0x564aae6aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38748/0 (socket says 172.21.15.50:38748) 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f914543a640 1 -- 172.21.15.50:0/3657342952 learned_addr learned my addr 172.21.15.50:0/3657342952 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.522+0000 7f9144c39640 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 0x564aae6a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 -- 172.21.15.50:0/3657342952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 msgr2=0x564aae6a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f91367fc640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 0x7f912c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 -- 172.21.15.50:0/3657342952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2df4d0 msgr2=0x564aae6aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2df4d0 0x564aae6aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 -- 172.21.15.50:0/3657342952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aae5cdd50 con 0x564aae3001b0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9145c3b640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 0x7f912c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f9144c39640 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 0x564aae6a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9130011700 tx=0x7f913000a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49de7b8db38a71c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f91367fc640 1 -- 172.21.15.50:0/2276859540 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9138103b20 con 0x7f9140103b70 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f912b7fe640 1 -- 172.21.15.50:0/3657342952 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130011260 con 0x564aae3001b0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f912b7fe640 1 -- 172.21.15.50:0/3657342952 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91300054b0 con 0x564aae3001b0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.523+0000 7f912b7fe640 1 -- 172.21.15.50:0/3657342952 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130004e60 con 0x564aae3001b0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 -- 172.21.15.50:0/3657342952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 msgr2=0x564aae6a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f9145c3b640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 0x7f912c0b4cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9138031920 tx=0x7f91380424d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 0x564aae6a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9130011700 tx=0x7f913000a2d0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 -- 172.21.15.50:0/3657342952 shutdown_connections 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2df4d0 0x564aae6aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 --2- 172.21.15.50:0/3657342952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae3001b0 0x564aae6a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 -- 172.21.15.50:0/3657342952 >> 172.21.15.50:0/3657342952 conn(0x564aae2f4300 msgr2=0x564aae69f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 -- 172.21.15.50:0/3657342952 shutdown_connections 2024-05-17T04:44:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.524+0000 7f914e638740 1 -- 172.21.15.50:0/3657342952 wait complete. 2024-05-17T04:44:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 -- start start 2024-05-17T04:44:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 Processor -- start 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564aae541ab0 con 0x564aae3001b0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.525+0000 7f914e638740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564aae5c24d0 con 0x564aae2d72f0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914e638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564aae6ab690 con 0x564aae2df4d0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914e638740 10 client.0 _ll_register_callbacks cb 0x564aae600950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9144c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9144c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9144c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38754/0 (socket says 172.21.15.50:38754) 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58942/0 (socket says 172.21.15.50:58942) 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9144c39640 1 -- 172.21.15.50:0/2603358865 learned_addr learned my addr 172.21.15.50:0/2603358865 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 -- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 msgr2=0x564aae6ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 -- 172.21.15.50:0/2603358865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 msgr2=0x564aae6b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 -- 172.21.15.50:0/2603358865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aae5c3340 con 0x564aae2df4d0 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9144c39640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f914543a640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x564aae69fe90 tx=0x7f913c012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9c94810544b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.526+0000 7f9128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00f8f0 con 0x564aae2df4d0 2024-05-17T04:44:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f913c00fa90 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91300054b0 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f913c00fc90 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 10 client.25666 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564aae36b790 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f913c00fe30 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f913c004b20 con 0x564aae2df4d0 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 10 client.25666 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 20 client.25666 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2bed80 0x564aae6bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 4 client.25666 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:44:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.527+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f91300054b0 con 0x564aae2bed80 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.528+0000 7f9144c39640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2bed80 0x564aae6bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.528+0000 7f9128ff9640 10 client.25666 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.528+0000 7f9144c39640 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2bed80 0x564aae6bad50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679fc6cb36eb0a37 server_cookie=d9902efa2bbb3391 in_seq=0 out_seq=0 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.529+0000 7f9128ff9640 1 -- 172.21.15.50:0/2603358865 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f91300054b0 con 0x564aae2bed80 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.529+0000 7f9128ff9640 10 client.25666 handle_command_reply: tid=0 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.529+0000 7f914e638740 1 client.25666 shutdown 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.529+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 msgr2=0x564aae6ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.529+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x564aae69fe90 tx=0x7f913c012bf0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 shutdown_connections 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2bed80 0x564aae6bad50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6ad610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 --2- 172.21.15.50:0/2603358865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2d72f0 0x564aae6ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 >> 172.21.15.50:0/2603358865 conn(0x564aae2f4300 msgr2=0x564aae6a2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f9128ff9640 0 client.25666 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 shutdown_connections 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 1 -- 172.21.15.50:0/2603358865 wait complete. 2024-05-17T04:44:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.530+0000 7f914e638740 20 client.25666 trim_cache size 0 max 16384 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 Processor -- start 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 -- start start 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 0x564aae6a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae3001b0 0x564aae6b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91401ae800 con 0x564aae2d72f0 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91401ac0d0 con 0x564aae3001b0 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f914e638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91401a9f20 con 0x564aae2df4d0 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.553+0000 7f9144c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 0x564aae6a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f9144c39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 0x564aae6a7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55676/0 (socket says 172.21.15.50:55676) 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f9144c39640 1 -- 172.21.15.50:0/3673925867 learned_addr learned my addr 172.21.15.50:0/3673925867 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58950/0 (socket says 172.21.15.50:58950) 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f9144c39640 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae3001b0 0x564aae6b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 -- 172.21.15.50:0/3673925867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae3001b0 msgr2=0x564aae6b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae3001b0 0x564aae6b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 -- 172.21.15.50:0/3673925867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 msgr2=0x564aae6a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 0x564aae6a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 -- 172.21.15.50:0/3673925867 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aae3e4620 con 0x564aae2df4d0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f914543a640 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f913c001eb0 tx=0x7f913c0123e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70852df4ad36e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f912affd640 1 -- 172.21.15.50:0/3673925867 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00f8f0 con 0x564aae2df4d0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.554+0000 7f912affd640 1 -- 172.21.15.50:0/3673925867 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f913c00fa90 con 0x564aae2df4d0 2024-05-17T04:44:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.555+0000 7f912affd640 1 -- 172.21.15.50:0/3673925867 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c003b50 con 0x564aae2df4d0 2024-05-17T04:44:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- 172.21.15.50:0/3673925867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 msgr2=0x564aae6a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f913c001eb0 tx=0x7f913c0123e0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- 172.21.15.50:0/3673925867 shutdown_connections 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae3001b0 0x564aae6b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2df4d0 0x564aae6a25e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 --2- 172.21.15.50:0/3673925867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae2d72f0 0x564aae6a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- 172.21.15.50:0/3673925867 >> 172.21.15.50:0/3673925867 conn(0x564aae2f4300 msgr2=0x564aae6a1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- 172.21.15.50:0/3673925867 shutdown_connections 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- 172.21.15.50:0/3673925867 wait complete. 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 -- start start 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.556+0000 7f914e638740 1 Processor -- start 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91401aa590 con 0x564aae3001b0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91401aba00 con 0x564aae2df4d0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91401aac60 con 0x564aae2d72f0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914e638740 10 client.0 _ll_register_callbacks cb 0x7f913c0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f9144c39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914543a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38778/0 (socket says 172.21.15.50:38778) 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f9144c39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58956/0 (socket says 172.21.15.50:58956) 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.557+0000 7f914543a640 1 -- 172.21.15.50:0/1924139809 learned_addr learned my addr 172.21.15.50:0/1924139809 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f9144c39640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 -- 172.21.15.50:0/1924139809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 msgr2=0x564aae6b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 -- 172.21.15.50:0/1924139809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 msgr2=0x564aae6b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 -- 172.21.15.50:0/1924139809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aae6b3db0 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f9144c39640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f914543a640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f913c012730 tx=0x7f913c003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20364d211954e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f9128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.558+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00f8f0 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.559+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f913c0051a0 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.559+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f913c00fa90 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.559+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f913c0053a0 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.559+0000 7f914e638740 10 client.25685 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.559+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564aae72a780 con 0x564aae2df4d0 2024-05-17T04:44:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.560+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f913c00fa90 con 0x564aae2df4d0 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.560+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f913c029c50 con 0x564aae2df4d0 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 10 client.25685 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 20 client.25685 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2941c0 0x564aae72cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 4 client.25685 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:44:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aae617c10 con 0x564aae2941c0 2024-05-17T04:44:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.561+0000 7f9144c39640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2941c0 0x564aae72cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.562+0000 7f9144c39640 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2941c0 0x564aae72cea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6b7cb14be5197 server_cookie=57d455bf8b43288b in_seq=0 out_seq=0 2024-05-17T04:44:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.562+0000 7f9128ff9640 10 client.25685 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f9128ff9640 1 -- 172.21.15.50:0/1924139809 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564aae617c10 con 0x564aae2941c0 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f9128ff9640 10 client.25685 handle_command_reply: tid=0 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 client.25685 shutdown 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 msgr2=0x564aae6b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f913c012730 tx=0x7f913c003e70 comp rx=0 tx=0).stop 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 shutdown_connections 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564aae2941c0 0x564aae72cea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564aae3001b0 0x564aae6b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564aae2df4d0 0x564aae6b31b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 --2- 172.21.15.50:0/1924139809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564aae2d72f0 0x564aae6b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f9128ff9640 0 client.25685 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 >> 172.21.15.50:0/1924139809 conn(0x564aae2f4300 msgr2=0x564aae6a6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 shutdown_connections 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 1 -- 172.21.15.50:0/1924139809 wait complete. 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.586+0000 7f914e638740 20 client.25685 trim_cache size 0 max 16384 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:44:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 msgr2=0x7f912c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 0x7f912c0b4cf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9138031920 tx=0x7f91380424d0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 msgr2=0x7f91401a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f913802ade0 tx=0x7f91380093d0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 shutdown_connections 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f912c0b27e0 0x7f912c0b4cf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9140103b70 0x7f91401a0ab0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9140103220 0x7f914019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 --2- 172.21.15.50:0/2276859540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9140102100 0x7f914006bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 >> 172.21.15.50:0/2276859540 conn(0x7f91400fd900 msgr2=0x7f91401064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 shutdown_connections 2024-05-17T04:44:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:15.587+0000 7f914769e640 1 -- 172.21.15.50:0/2276859540 wait complete. 2024-05-17T04:44:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:15 smithi105 ceph-mon[34795]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:44:16.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:15 smithi110 ceph-mon[35497]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:44:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:15 smithi050 ceph-mon[31739]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:44:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 320.0 s, 115.6 tps, lat 307.741 ms stddev 79.931 2024-05-17T04:44:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:17 smithi105 ceph-mon[34795]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-05-17T04:44:18.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:17 smithi110 ceph-mon[35497]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-05-17T04:44:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:17 smithi050 ceph-mon[31739]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-05-17T04:44:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:19 smithi105 ceph-mon[34795]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-05-17T04:44:20.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:19 smithi110 ceph-mon[35497]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-05-17T04:44:20.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:19 smithi050 ceph-mon[31739]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.3 MiB/s wr, 277 op/s 2024-05-17T04:44:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:22 smithi105 ceph-mon[34795]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-05-17T04:44:22.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:22 smithi110 ceph-mon[35497]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-05-17T04:44:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:22 smithi050 ceph-mon[31739]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-05-17T04:44:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 325.0 s, 117.8 tps, lat 306.392 ms stddev 67.989 2024-05-17T04:44:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:24 smithi105 ceph-mon[34795]: pgmap v681: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:24.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:24 smithi110 ceph-mon[35497]: pgmap v681: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:24.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:24 smithi050 ceph-mon[31739]: pgmap v681: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:26.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:26 smithi105 ceph-mon[34795]: pgmap v682: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-17T04:44:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:26 smithi110 ceph-mon[35497]: pgmap v682: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-17T04:44:26.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:26 smithi050 ceph-mon[31739]: pgmap v682: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.6 MiB/s wr, 330 op/s 2024-05-17T04:44:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 330.0 s, 104.4 tps, lat 341.963 ms stddev 85.230 2024-05-17T04:44:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:28 smithi105 ceph-mon[34795]: pgmap v683: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.5 MiB/s wr, 293 op/s 2024-05-17T04:44:28.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:28 smithi110 ceph-mon[35497]: pgmap v683: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.5 MiB/s wr, 293 op/s 2024-05-17T04:44:28.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:28 smithi050 ceph-mon[31739]: pgmap v683: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.5 MiB/s wr, 293 op/s 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 -- 172.21.15.50:0/2421309243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 msgr2=0x7f619410b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 --2- 172.21.15.50:0/2421309243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f619410b6e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f618c009cd0 tx=0x7f618c02a8e0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 -- 172.21.15.50:0/2421309243 shutdown_connections 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 --2- 172.21.15.50:0/2421309243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f619410b6e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 --2- 172.21.15.50:0/2421309243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 --2- 172.21.15.50:0/2421309243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 0x7f6194104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.710+0000 7f6199c09640 1 -- 172.21.15.50:0/2421309243 >> 172.21.15.50:0/2421309243 conn(0x7f61940fd010 msgr2=0x7f61940ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.711+0000 7f6199c09640 1 -- 172.21.15.50:0/2421309243 shutdown_connections 2024-05-17T04:44:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.711+0000 7f6199c09640 1 -- 172.21.15.50:0/2421309243 wait complete. 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 Processor -- start 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 -- start start 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f6194199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 0x7f61941a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f619410eb70 con 0x7f6194104730 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f619410e9f0 con 0x7f6194101170 2024-05-17T04:44:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.712+0000 7f6199c09640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f619410ecf0 con 0x7f6194101ac0 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f6194199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f6194199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48488/0 (socket says 172.21.15.50:48488) 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f6192ffd640 1 -- 172.21.15.50:0/1536221807 learned_addr learned my addr 172.21.15.50:0/1536221807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f6193fff640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 0x7f61941a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 -- 172.21.15.50:0/1536221807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 msgr2=0x7f6194199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f6194199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 -- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 msgr2=0x7f61941a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 0x7f61941a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 -- 172.21.15.50:0/1536221807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61941a1350 con 0x7f6194101170 2024-05-17T04:44:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.713+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194199810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f618400e9c0 tx=0x7f618400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.714+0000 7f6190ff9640 1 -- 172.21.15.50:0/1536221807 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618400ccc0 con 0x7f6194101170 2024-05-17T04:44:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.714+0000 7f6190ff9640 1 -- 172.21.15.50:0/1536221807 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61840108c0 con 0x7f6194101170 2024-05-17T04:44:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.714+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61941a1640 con 0x7f6194101170 2024-05-17T04:44:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.714+0000 7f6190ff9640 1 -- 172.21.15.50:0/1536221807 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184018730 con 0x7f6194101170 2024-05-17T04:44:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.714+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941a1b90 con 0x7f6194101170 2024-05-17T04:44:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 Processor -- start 2024-05-17T04:44:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 -- start start 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 0x55bd9ce0fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bd9cd7acc0 con 0x55bd9ca444d0 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bd9cd6a860 con 0x55bd9ca651b0 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bd9cd6e8a0 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f6193fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 0x55bd9ce0fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f6193fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49412/0 (socket says 172.21.15.50:49412) 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f6193fff640 1 -- 172.21.15.50:0/1774147033 learned_addr learned my addr 172.21.15.50:0/1774147033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.717+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 0x55bd9ce0fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56520/0 (socket says 172.21.15.50:56520) 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6190ff9640 1 -- 172.21.15.50:0/1536221807 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6184010430 con 0x7f6194101170 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 -- 172.21.15.50:0/1774147033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca3c2f0 msgr2=0x55bd9ce08cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 -- 172.21.15.50:0/1774147033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 msgr2=0x55bd9ce0fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 0x55bd9ce0fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 -- 172.21.15.50:0/1774147033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9cd32d50 con 0x55bd9ca651b0 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.718+0000 7f6193fff640 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f618c009cd0 tx=0x7f618c031430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ff523ceee352cc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.719+0000 7f6179ffb640 1 -- 172.21.15.50:0/1774147033 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c048070 con 0x55bd9ca651b0 2024-05-17T04:44:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.719+0000 7f6179ffb640 1 -- 172.21.15.50:0/1774147033 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618c036070 con 0x55bd9ca651b0 2024-05-17T04:44:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.719+0000 7f6179ffb640 1 -- 172.21.15.50:0/1774147033 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c0043c0 con 0x55bd9ca651b0 2024-05-17T04:44:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.719+0000 7f6190ff9640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 0x7f617c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f6190ff9640 1 -- 172.21.15.50:0/1536221807 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f61840e28c0 con 0x7f6194101170 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 0x7f617c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1774147033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 msgr2=0x55bd9ce08770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f618c009cd0 tx=0x7f618c031430 comp rx=0 tx=0).stop 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1774147033 shutdown_connections 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca444d0 0x55bd9ce0fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce08cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1774147033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce08770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1774147033 >> 172.21.15.50:0/1774147033 conn(0x55bd9ca59300 msgr2=0x55bd9ce041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61937fe640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 0x7f617c0b4cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f61840a9b30 tx=0x7f61840a94f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.720+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1774147033 shutdown_connections 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.721+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1774147033 wait complete. 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.721+0000 7f61a0b9f740 1 -- start start 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.721+0000 7f61a0b9f740 1 Processor -- start 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bd9cca6ab0 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bd9cd274d0 con 0x55bd9ca651b0 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55bd9ce10630 con 0x55bd9ca444d0 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f61a0b9f740 10 client.0 _ll_register_callbacks cb 0x55bd9cd65950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48508/0 (socket says 172.21.15.50:48508) 2024-05-17T04:44:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 -- 172.21.15.50:0/2533319259 learned_addr learned my addr 172.21.15.50:0/2533319259 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6193fff640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 -- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 msgr2=0x55bd9ce19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 -- 172.21.15.50:0/2533319259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 msgr2=0x55bd9ce11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.722+0000 7f6192ffd640 1 -- 172.21.15.50:0/2533319259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd9cd28340 con 0x55bd9ca444d0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6192ffd640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6188012790 tx=0x7f6188012c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea3fc1b45695388 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618800f920 con 0x55bd9ca444d0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f618800fac0 con 0x55bd9ca444d0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c036070 con 0x55bd9ca444d0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6188025050 con 0x55bd9ca444d0 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f61a0b9f740 10 client.25681 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bd9cad0790 con 0x55bd9ca444d0 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.723+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618c036070 con 0x55bd9ca444d0 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6188010430 con 0x55bd9ca444d0 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 10 client.25681 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 20 client.25681 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9ca23d80 0x55bd9ce1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 4 client.25681 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd9cd7cc10 con 0x55bd9ca23d80 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f6193fff640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9ca23d80 0x55bd9ce1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f6193fff640 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9ca23d80 0x55bd9ce1f890 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37528b2dbb158158 server_cookie=bc4a7b7e8abc3c65 in_seq=0 out_seq=0 2024-05-17T04:44:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.724+0000 7f6172ffd640 10 client.25681 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.725+0000 7f6172ffd640 1 -- 172.21.15.50:0/2533319259 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd9cd7cc10 con 0x55bd9ca23d80 2024-05-17T04:44:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.725+0000 7f6172ffd640 10 client.25681 handle_command_reply: tid=0 2024-05-17T04:44:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.725+0000 7f61a0b9f740 1 client.25681 shutdown 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 msgr2=0x55bd9ce12520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6188012790 tx=0x7f6188012c50 comp rx=0 tx=0).stop 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 shutdown_connections 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce12520 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9ca23d80 0x55bd9ce1f890 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/2533319259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 >> 172.21.15.50:0/2533319259 conn(0x55bd9ca59300 msgr2=0x55bd9ce031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f6172ffd640 0 client.25681 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 shutdown_connections 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 1 -- 172.21.15.50:0/2533319259 wait complete. 2024-05-17T04:44:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.726+0000 7f61a0b9f740 20 client.25681 trim_cache size 0 max 16384 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.748+0000 7f61a0b9f740 1 Processor -- start 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.748+0000 7f61a0b9f740 1 -- start start 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61941b0a40 con 0x55bd9ca651b0 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ae310 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ac160 con 0x55bd9ca444d0 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6192ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48524/0 (socket says 172.21.15.50:48524) 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49424/0 (socket says 172.21.15.50:49424) 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6192ffd640 1 -- 172.21.15.50:0/780663161 learned_addr learned my addr 172.21.15.50:0/780663161 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 -- 172.21.15.50:0/780663161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 msgr2=0x55bd9ce193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 -- 172.21.15.50:0/780663161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca651b0 msgr2=0x55bd9ce19930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6193fff640 1 -- 172.21.15.50:0/780663161 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd9cb49620 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.749+0000 7f6192ffd640 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.750+0000 7f6193fff640 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f618c004af0 tx=0x7f618c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e7e94ea16f538f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.750+0000 7f61797fa640 1 -- 172.21.15.50:0/780663161 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c048040 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.750+0000 7f61797fa640 1 -- 172.21.15.50:0/780663161 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618c036040 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.750+0000 7f61797fa640 1 -- 172.21.15.50:0/780663161 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c004620 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 -- 172.21.15.50:0/780663161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 msgr2=0x55bd9ce04200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f618c004af0 tx=0x7f618c000f40 comp rx=0 tx=0).stop 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 -- 172.21.15.50:0/780663161 shutdown_connections 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca651b0 0x55bd9ce19930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca444d0 0x55bd9ce193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/780663161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce04200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 -- 172.21.15.50:0/780663161 >> 172.21.15.50:0/780663161 conn(0x55bd9ca59300 msgr2=0x55bd9ce03af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 -- 172.21.15.50:0/780663161 shutdown_connections 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.751+0000 7f61a0b9f740 1 -- 172.21.15.50:0/780663161 wait complete. 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.752+0000 7f61a0b9f740 1 -- start start 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.752+0000 7f61a0b9f740 1 Processor -- start 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.752+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.752+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 0x55bd9ce196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.752+0000 7f6193fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f61a0b9f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca651b0 0x55bd9ce147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ac7d0 con 0x55bd9ca3c2f0 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f61a0b9f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61941adc40 con 0x55bd9ca444d0 2024-05-17T04:44:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6193fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56534/0 (socket says 172.21.15.50:56534) 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6193fff640 1 -- 172.21.15.50:0/1010023647 learned_addr learned my addr 172.21.15.50:0/1010023647 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61941acea0 con 0x55bd9ca651b0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 0x55bd9ce196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f61a0b9f740 10 client.0 _ll_register_callbacks cb 0x7f6188009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 -- 172.21.15.50:0/1010023647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca651b0 msgr2=0x55bd9ce147d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca651b0 0x55bd9ce147d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 -- 172.21.15.50:0/1010023647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 msgr2=0x55bd9ce14290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 -- 172.21.15.50:0/1010023647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd9ce8e400 con 0x55bd9ca444d0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6193fff640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6192ffd640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 0x55bd9ce196d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6188001eb0 tx=0x7f61880106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb988350a54bea91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.753+0000 7f6172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:44:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188022040 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f618800f920 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c036040 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6188010c80 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f61a0b9f740 10 client.25700 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bd9ce8f560 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618c036040 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.754+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6188004680 con 0x55bd9ca444d0 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 10 client.25700 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 20 client.25700 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9c9f91c0 0x55bd9ce91f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 4 client.25700 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:44:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd9cd763f0 con 0x55bd9c9f91c0 2024-05-17T04:44:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f6193fff640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9c9f91c0 0x55bd9ce91f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f6172ffd640 10 client.25700 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.755+0000 7f6193fff640 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9c9f91c0 0x55bd9ce91f70 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2126e7d881cbd2 server_cookie=aeadecfbfe8e3536 in_seq=0 out_seq=0 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.775+0000 7f6172ffd640 1 -- 172.21.15.50:0/1010023647 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bd9cd763f0 con 0x55bd9c9f91c0 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.775+0000 7f6172ffd640 10 client.25700 handle_command_reply: tid=0 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.775+0000 7f61a0b9f740 1 client.25700 shutdown 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.775+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 msgr2=0x55bd9ce196d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.775+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 0x55bd9ce196d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6188001eb0 tx=0x7f61880106e0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 shutdown_connections 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55bd9ca651b0 0x55bd9ce147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bd9ca444d0 0x55bd9ce196d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55bd9c9f91c0 0x55bd9ce91f70 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 --2- 172.21.15.50:0/1010023647 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bd9ca3c2f0 0x55bd9ce14290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f6172ffd640 0 client.25700 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 >> 172.21.15.50:0/1010023647 conn(0x55bd9ca59300 msgr2=0x55bd9ce027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 shutdown_connections 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 1 -- 172.21.15.50:0/1010023647 wait complete. 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.776+0000 7f61a0b9f740 20 client.25700 trim_cache size 0 max 16384 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:44:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 msgr2=0x7f617c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 0x7f617c0b4cf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f61840a9b30 tx=0x7f61840a94f0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 msgr2=0x7f6194199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194199810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f618400e9c0 tx=0x7f618400ee80 comp rx=0 tx=0).stop 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 shutdown_connections 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f617c0b27e0 0x7f617c0b4cf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6194104730 0x7f61941a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6194101ac0 0x7f6194199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 --2- 172.21.15.50:0/1536221807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6194101170 0x7f6194199810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 >> 172.21.15.50:0/1536221807 conn(0x7f61940fd010 msgr2=0x7f6194107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 shutdown_connections 2024-05-17T04:44:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:28.777+0000 7f6199c09640 1 -- 172.21.15.50:0/1536221807 wait complete. 2024-05-17T04:44:30.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:30 smithi105 ceph-mon[34795]: pgmap v684: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-17T04:44:30.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:30 smithi110 ceph-mon[35497]: pgmap v684: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-17T04:44:30.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:30 smithi050 ceph-mon[31739]: pgmap v684: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-05-17T04:44:32.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:32 smithi105 ceph-mon[34795]: pgmap v685: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-05-17T04:44:32.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:32 smithi110 ceph-mon[35497]: pgmap v685: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-05-17T04:44:32.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:32 smithi050 ceph-mon[31739]: pgmap v685: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.6 MiB/s wr, 320 op/s 2024-05-17T04:44:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 335.0 s, 101.8 tps, lat 351.960 ms stddev 85.588 2024-05-17T04:44:34.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:34 smithi105 ceph-mon[34795]: pgmap v686: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-05-17T04:44:34.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:34 smithi110 ceph-mon[35497]: pgmap v686: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-05-17T04:44:34.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:34 smithi050 ceph-mon[31739]: pgmap v686: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.6 MiB/s wr, 293 op/s 2024-05-17T04:44:35.304 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:35 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:44:35.317 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:35 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:44:35.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:35 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:44:36.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:36 smithi105 ceph-mon[34795]: pgmap v687: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-05-17T04:44:36.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:36 smithi110 ceph-mon[35497]: pgmap v687: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-05-17T04:44:36.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:36 smithi050 ceph-mon[31739]: pgmap v687: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-05-17T04:44:37.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:37 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:37 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:37 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 340.0 s, 116.2 tps, lat 313.479 ms stddev 108.077 2024-05-17T04:44:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:38 smithi105 ceph-mon[34795]: pgmap v688: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-17T04:44:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:44:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:38 smithi110 ceph-mon[35497]: pgmap v688: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-17T04:44:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:44:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:44:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:38 smithi050 ceph-mon[31739]: pgmap v688: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-05-17T04:44:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:44:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:44:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:44:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:40 smithi105 ceph-mon[34795]: pgmap v689: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-05-17T04:44:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:40 smithi110 ceph-mon[35497]: pgmap v689: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-05-17T04:44:40.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:40 smithi050 ceph-mon[31739]: pgmap v689: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 265 op/s 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 -- 172.21.15.50:0/759890997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 msgr2=0x7fae0c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 --2- 172.21.15.50:0/759890997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c10a400 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fae04009ca0 tx=0x7fae0402a8c0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 -- 172.21.15.50:0/759890997 shutdown_connections 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 --2- 172.21.15.50:0/759890997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c10a400 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 --2- 172.21.15.50:0/759890997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c1031b0 0x7fae0c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 --2- 172.21.15.50:0/759890997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c102090 0x7fae0c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.896+0000 7fae144a7640 1 -- 172.21.15.50:0/759890997 >> 172.21.15.50:0/759890997 conn(0x7fae0c0fd8b0 msgr2=0x7fae0c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.897+0000 7fae144a7640 1 -- 172.21.15.50:0/759890997 shutdown_connections 2024-05-17T04:44:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.897+0000 7fae144a7640 1 -- 172.21.15.50:0/759890997 wait complete. 2024-05-17T04:44:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.897+0000 7fae144a7640 1 Processor -- start 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.897+0000 7fae144a7640 1 -- start start 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10dac0 con 0x7fae0c103b00 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10d940 con 0x7fae0c1031b0 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae144a7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10dc40 con 0x7fae0c102090 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae11a42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12a44640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae11a42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54368/0 (socket says 172.21.15.50:54368) 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51558/0 (socket says 172.21.15.50:51558) 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae11a42640 1 -- 172.21.15.50:0/4033825982 learned_addr learned my addr 172.21.15.50:0/4033825982 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 -- 172.21.15.50:0/4033825982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 msgr2=0x7fae0c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 -- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 msgr2=0x7fae0c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 -- 172.21.15.50:0/4033825982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c1a12d0 con 0x7fae0c102090 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae11a42640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.898+0000 7fae12243640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae0800e9c0 tx=0x7fae0800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.899+0000 7fae037fe640 1 -- 172.21.15.50:0/4033825982 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800cd80 con 0x7fae0c102090 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.899+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c1a15c0 con 0x7fae0c102090 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.899+0000 7fae037fe640 1 -- 172.21.15.50:0/4033825982 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae080035f0 con 0x7fae0c102090 2024-05-17T04:44:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.899+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a1b60 con 0x7fae0c102090 2024-05-17T04:44:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.899+0000 7fae037fe640 1 -- 172.21.15.50:0/4033825982 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08010690 con 0x7fae0c102090 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 Processor -- start 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 -- start start 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef7b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef7bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55895eeedcc0 con 0x55895ebb74d0 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55895eedd860 con 0x55895ebaf2f0 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae1affd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55895eee18a0 con 0x55895ebd81b0 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae11a42640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae11a42640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37464/0 (socket says 172.21.15.50:37464) 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae12a44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef7b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.902+0000 7fae11a42640 1 -- 172.21.15.50:0/226517207 learned_addr learned my addr 172.21.15.50:0/226517207 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae037fe640 1 -- 172.21.15.50:0/4033825982 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae08004000 con 0x7fae0c102090 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 -- 172.21.15.50:0/226517207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 msgr2=0x55895ef7b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef7b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 -- 172.21.15.50:0/226517207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 msgr2=0x55895ef7bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef7bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 -- 172.21.15.50:0/226517207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895eea5d50 con 0x55895ebb74d0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae11a42640 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0115f0 tx=0x7fadfc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f8182bdc15d3c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae00bfe640 1 -- 172.21.15.50:0/226517207 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc005020 con 0x55895ebb74d0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae00bfe640 1 -- 172.21.15.50:0/226517207 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadfc0047a0 con 0x55895ebb74d0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.903+0000 7fae00bfe640 1 -- 172.21.15.50:0/226517207 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00f920 con 0x55895ebb74d0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae037fe640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 0x7fadf80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae12a44640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 0x7fadf80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae037fe640 1 -- 172.21.15.50:0/4033825982 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae080e2820 con 0x7fae0c102090 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 -- 172.21.15.50:0/226517207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 msgr2=0x55895ef82c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0115f0 tx=0x7fadfc011ab0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae12a44640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 0x7fadf80b4cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55895ef7c960 tx=0x7fae04031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 -- 172.21.15.50:0/226517207 shutdown_connections 2024-05-17T04:44:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef82c10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef7bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 --2- 172.21.15.50:0/226517207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 -- 172.21.15.50:0/226517207 >> 172.21.15.50:0/226517207 conn(0x55895ebcc300 msgr2=0x55895ef771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 -- 172.21.15.50:0/226517207 shutdown_connections 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.904+0000 7fae1affd740 1 -- 172.21.15.50:0/226517207 wait complete. 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 -- start start 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 Processor -- start 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55895ee19ab0 con 0x55895ebb74d0 2024-05-17T04:44:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55895ee9a4d0 con 0x55895ebaf2f0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55895ef83690 con 0x55895ebd81b0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae1affd740 10 client.0 _ll_register_callbacks cb 0x55895eed8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae11a42640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae12243640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae12243640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:54380/0 (socket says 172.21.15.50:54380) 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae12243640 1 -- 172.21.15.50:0/3531154837 learned_addr learned my addr 172.21.15.50:0/3531154837 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.905+0000 7fae12a44640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 -- 172.21.15.50:0/3531154837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 msgr2=0x55895ef8c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef8c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 -- 172.21.15.50:0/3531154837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 msgr2=0x55895ef85600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef85600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 -- 172.21.15.50:0/3531154837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895ee9b340 con 0x55895ebaf2f0 2024-05-17T04:44:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12a44640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae12243640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae080a6d20 tx=0x7fae080a95c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07e0ad7e3fe1a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fadf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080a9ab0 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae080a60a0 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadfc0047a0 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fae080ade60 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.906+0000 7fae1affd740 10 client.25709 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55895ec43790 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadfc0047a0 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fae080a5d40 con 0x55895ebaf2f0 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 10 client.25709 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 20 client.25709 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895eb96d80 0x55895ef92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 4 client.25709 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:44:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895eeefc10 con 0x55895eb96d80 2024-05-17T04:44:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.907+0000 7fae11a42640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895eb96d80 0x55895ef92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.908+0000 7fae11a42640 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895eb96d80 0x55895ef92810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92af5a882240ded server_cookie=5ba6ff48f2e98ab6 in_seq=0 out_seq=0 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.908+0000 7fadf5ffb640 10 client.25709 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fadf5ffb640 1 -- 172.21.15.50:0/3531154837 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55895eeefc10 con 0x55895eb96d80 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fadf5ffb640 10 client.25709 handle_command_reply: tid=0 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 client.25709 shutdown 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 msgr2=0x55895ef850c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae080a6d20 tx=0x7fae080a95c0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 shutdown_connections 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebd81b0 0x55895ef8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895eb96d80 0x55895ef92810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebb74d0 0x55895ef85600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 --2- 172.21.15.50:0/3531154837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebaf2f0 0x55895ef850c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 >> 172.21.15.50:0/3531154837 conn(0x55895ebcc300 msgr2=0x55895ef761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fadf5ffb640 0 client.25709 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 shutdown_connections 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.909+0000 7fae1affd740 1 -- 172.21.15.50:0/3531154837 wait complete. 2024-05-17T04:44:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.910+0000 7fae1affd740 20 client.25709 trim_cache size 0 max 16384 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 Processor -- start 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 -- start start 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1ac550 con 0x55895ebaf2f0 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1a9e20 con 0x55895ebd81b0 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae1affd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1a7c70 con 0x55895ebb74d0 2024-05-17T04:44:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae12243640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae11a42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae12243640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37478/0 (socket says 172.21.15.50:37478) 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae11a42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51586/0 (socket says 172.21.15.50:51586) 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae12243640 1 -- 172.21.15.50:0/1122423338 learned_addr learned my addr 172.21.15.50:0/1122423338 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.931+0000 7fae12a44640 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 -- 172.21.15.50:0/1122423338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 msgr2=0x55895ef8c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 -- 172.21.15.50:0/1122423338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 msgr2=0x55895ef79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 -- 172.21.15.50:0/1122423338 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895ecbc620 con 0x55895ebaf2f0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae11a42640 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fae12243640 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae080a5b30 tx=0x7fae08021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e00911376685128 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fadf7fff640 1 -- 172.21.15.50:0/1122423338 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080a9ab0 con 0x55895ebaf2f0 2024-05-17T04:44:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fadf7fff640 1 -- 172.21.15.50:0/1122423338 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae080b2910 con 0x55895ebaf2f0 2024-05-17T04:44:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.932+0000 7fadf7fff640 1 -- 172.21.15.50:0/1122423338 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08003bb0 con 0x55895ebaf2f0 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 -- 172.21.15.50:0/1122423338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 msgr2=0x55895ef77270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae080a5b30 tx=0x7fae08021000 comp rx=0 tx=0).stop 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 -- 172.21.15.50:0/1122423338 shutdown_connections 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 --2- 172.21.15.50:0/1122423338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895ebaf2f0 0x55895ef77270 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 -- 172.21.15.50:0/1122423338 >> 172.21.15.50:0/1122423338 conn(0x55895ebcc300 msgr2=0x55895ef76b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 -- 172.21.15.50:0/1122423338 shutdown_connections 2024-05-17T04:44:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.933+0000 7fae1affd740 1 -- 172.21.15.50:0/1122423338 wait complete. 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 -- start start 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 Processor -- start 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1a82e0 con 0x55895eb6c1c0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1a9750 con 0x55895ebd81b0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1a89b0 con 0x55895ebb74d0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae1affd740 10 client.0 _ll_register_callbacks cb 0x7fae0800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae12a44640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae12243640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae12a44640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:37480/0 (socket says 172.21.15.50:37480) 2024-05-17T04:44:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae12a44640 1 -- 172.21.15.50:0/366549851 learned_addr learned my addr 172.21.15.50:0/366549851 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.934+0000 7fae11a42640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 -- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 msgr2=0x55895ef88e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef88e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 -- 172.21.15.50:0/366549851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 msgr2=0x55895ef893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 -- 172.21.15.50:0/366549851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895eff8b90 con 0x55895ebb74d0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12a44640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae12243640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55895ef84c60 tx=0x7fae080a95c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4113d0397348a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fadf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080a9ab0 con 0x55895ebb74d0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae080b2910 con 0x55895ebb74d0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae080a60a0 con 0x55895ebb74d0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fae080a8d10 con 0x55895ebb74d0 2024-05-17T04:44:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae1affd740 10 client.25702 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55895eff8b90 con 0x55895ebb74d0 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.935+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae080b2910 con 0x55895ebb74d0 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fae080a8810 con 0x55895ebb74d0 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 10 client.25702 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 20 client.25702 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895effa270 0x55895effc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 4 client.25702 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55895eee93f0 con 0x55895effa270 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.936+0000 7fae11a42640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895effa270 0x55895effc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.937+0000 7fadf5ffb640 10 client.25702 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.937+0000 7fae11a42640 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895effa270 0x55895effc690 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac57f256d29adb3 server_cookie=7f28e8feb112f6e in_seq=0 out_seq=0 2024-05-17T04:44:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fadf5ffb640 1 -- 172.21.15.50:0/366549851 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55895eee93f0 con 0x55895effa270 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fadf5ffb640 10 client.25702 handle_command_reply: tid=0 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 client.25702 shutdown 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 msgr2=0x55895ef8c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55895ef84c60 tx=0x7fae080a95c0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 shutdown_connections 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55895effa270 0x55895effc690 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55895eb6c1c0 0x55895ef893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fadf5ffb640 0 client.25702 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55895ebd81b0 0x55895ef88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 --2- 172.21.15.50:0/366549851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55895ebb74d0 0x55895ef8c300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 >> 172.21.15.50:0/366549851 conn(0x55895ebcc300 msgr2=0x55895ef757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 shutdown_connections 2024-05-17T04:44:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.951+0000 7fae1affd740 1 -- 172.21.15.50:0/366549851 wait complete. 2024-05-17T04:44:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae1affd740 20 client.25702 trim_cache size 0 max 16384 2024-05-17T04:44:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:44:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:44:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 msgr2=0x7fadf80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 0x7fadf80b4cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55895ef7c960 tx=0x7fae04031000 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 msgr2=0x7fae0c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae0800e9c0 tx=0x7fae0800ee80 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 shutdown_connections 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fadf80b27e0 0x7fadf80b4cf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae0c103b00 0x7fae0c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae0c1031b0 0x7fae0c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 --2- 172.21.15.50:0/4033825982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae0c102090 0x7fae0c199790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 >> 172.21.15.50:0/4033825982 conn(0x7fae0c0fd8b0 msgr2=0x7fae0c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 shutdown_connections 2024-05-17T04:44:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:41.952+0000 7fae144a7640 1 -- 172.21.15.50:0/4033825982 wait complete. 2024-05-17T04:44:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:42 smithi105 ceph-mon[34795]: pgmap v690: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-05-17T04:44:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:42 smithi110 ceph-mon[35497]: pgmap v690: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-05-17T04:44:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:42 smithi050 ceph-mon[31739]: pgmap v690: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-05-17T04:44:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 345.0 s, 111.6 tps, lat 321.454 ms stddev 73.048 2024-05-17T04:44:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:44 smithi105 ceph-mon[34795]: pgmap v691: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-05-17T04:44:44.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:44 smithi110 ceph-mon[35497]: pgmap v691: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-05-17T04:44:44.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:44 smithi050 ceph-mon[31739]: pgmap v691: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 264 op/s 2024-05-17T04:44:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:46 smithi105 ceph-mon[34795]: pgmap v692: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-17T04:44:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:46 smithi110 ceph-mon[35497]: pgmap v692: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-17T04:44:46.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:46 smithi050 ceph-mon[31739]: pgmap v692: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-17T04:44:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 350.0 s, 112.0 tps, lat 319.853 ms stddev 132.017 2024-05-17T04:44:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:48 smithi105 ceph-mon[34795]: pgmap v693: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-05-17T04:44:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:48 smithi110 ceph-mon[35497]: pgmap v693: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-05-17T04:44:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:48 smithi050 ceph-mon[31739]: pgmap v693: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.0 MiB/s wr, 238 op/s 2024-05-17T04:44:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:50 smithi105 ceph-mon[34795]: pgmap v694: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-17T04:44:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:50 smithi110 ceph-mon[35497]: pgmap v694: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-17T04:44:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:50 smithi050 ceph-mon[31739]: pgmap v694: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-17T04:44:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:52 smithi105 ceph-mon[34795]: pgmap v695: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:44:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:52 smithi110 ceph-mon[35497]: pgmap v695: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:44:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:52 smithi050 ceph-mon[31739]: pgmap v695: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:44:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 355.0 s, 115.8 tps, lat 311.384 ms stddev 106.763 2024-05-17T04:44:53.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:53 smithi105 ceph-mon[34795]: pgmap v696: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.0 MiB/s wr, 247 op/s 2024-05-17T04:44:53.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:53 smithi110 ceph-mon[35497]: pgmap v696: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.0 MiB/s wr, 247 op/s 2024-05-17T04:44:53.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:53 smithi050 ceph-mon[31739]: pgmap v696: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 2.0 MiB/s wr, 247 op/s 2024-05-17T04:44:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.071+0000 7f5b1984e640 1 -- 172.21.15.50:0/3269691006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b141042a0 msgr2=0x7f5b141046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.071+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3269691006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b141042a0 0x7f5b141046b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009900 tx=0x7f5b0802a410 comp rx=0 tx=0).stop 2024-05-17T04:44:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.071+0000 7f5b1984e640 1 -- 172.21.15.50:0/3269691006 shutdown_connections 2024-05-17T04:44:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3269691006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b14104bf0 0x7f5b1410b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3269691006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b141042a0 0x7f5b141046b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3269691006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 0x7f5b14103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 -- 172.21.15.50:0/3269691006 >> 172.21.15.50:0/3269691006 conn(0x7f5b140fe980 msgr2=0x7f5b14100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 -- 172.21.15.50:0/3269691006 shutdown_connections 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 -- 172.21.15.50:0/3269691006 wait complete. 2024-05-17T04:44:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.072+0000 7f5b1984e640 1 Processor -- start 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 -- start start 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 0x7f5b140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 0x7f5b14075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410eb60 con 0x7f5b14104bf0 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410e9e0 con 0x7f5b141042a0 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b1984e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410ece0 con 0x7f5b14103180 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51398/0 (socket says 172.21.15.50:51398) 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 0x7f5b14075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b137fe640 1 -- 172.21.15.50:0/3817356344 learned_addr learned my addr 172.21.15.50:0/3817356344 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.073+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 0x7f5b140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 -- 172.21.15.50:0/3817356344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 msgr2=0x7f5b140769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 0x7f5b140769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 -- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 msgr2=0x7f5b140755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 -- 172.21.15.50:0/3817356344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b14075d00 con 0x7f5b141042a0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b137fe640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 0x7f5b14075090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005760 tx=0x7f5b08005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b1884c640 1 -- 172.21.15.50:0/3817356344 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0803f070 con 0x7f5b141042a0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b141a3900 con 0x7f5b141042a0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b141a3d60 con 0x7f5b141042a0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b1884c640 1 -- 172.21.15.50:0/3817356344 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b08003a80 con 0x7f5b141042a0 2024-05-17T04:44:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.074+0000 7f5b1884c640 1 -- 172.21.15.50:0/3817356344 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08035070 con 0x7f5b141042a0 2024-05-17T04:44:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 Processor -- start 2024-05-17T04:44:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 -- start start 2024-05-17T04:44:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029e31b0 0x555702d86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555702cf8cc0 con 0x5557029ba2f0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555702ce8860 con 0x5557029e31b0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b207ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555702cec8a0 con 0x5557029c24d0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029e31b0 0x555702d86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51414/0 (socket says 172.21.15.50:51414) 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.077+0000 7f5b12ffd640 1 -- 172.21.15.50:0/858804934 learned_addr learned my addr 172.21.15.50:0/858804934 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 -- 172.21.15.50:0/858804934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 msgr2=0x555702d8dc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d8dc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 -- 172.21.15.50:0/858804934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029e31b0 msgr2=0x555702d86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029e31b0 0x555702d86770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 -- 172.21.15.50:0/858804934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555702cb0d50 con 0x5557029ba2f0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b1884c640 1 -- 172.21.15.50:0/3817356344 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5b08043400 con 0x7f5b141042a0 2024-05-17T04:44:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009c90 tx=0x7f5b04011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbc8a643353ec4e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5af9ffb640 1 -- 172.21.15.50:0/858804934 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005220 con 0x5557029ba2f0 2024-05-17T04:44:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.078+0000 7f5af9ffb640 1 -- 172.21.15.50:0/858804934 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b04009e40 con 0x5557029ba2f0 2024-05-17T04:44:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5af9ffb640 1 -- 172.21.15.50:0/858804934 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040197a0 con 0x5557029ba2f0 2024-05-17T04:44:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b1884c640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 0x7f5afc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b137fe640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 0x7f5afc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 -- 172.21.15.50:0/858804934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 msgr2=0x555702d86cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009c90 tx=0x7f5b04011810 comp rx=0 tx=0).stop 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b1884c640 1 -- 172.21.15.50:0/3817356344 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b08102550 con 0x7f5b141042a0 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 -- 172.21.15.50:0/858804934 shutdown_connections 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d86cf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 --2- 172.21.15.50:0/858804934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029e31b0 0x555702d86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.079+0000 7f5b207ee740 1 -- 172.21.15.50:0/858804934 >> 172.21.15.50:0/858804934 conn(0x5557029d7300 msgr2=0x555702d821a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.080+0000 7f5b207ee740 1 -- 172.21.15.50:0/858804934 shutdown_connections 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.080+0000 7f5b137fe640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 0x7f5afc0b4d70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x555702d87960 tx=0x7f5b0c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.080+0000 7f5b207ee740 1 -- 172.21.15.50:0/858804934 wait complete. 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.080+0000 7f5b207ee740 1 -- start start 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 Processor -- start 2024-05-17T04:44:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d90640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029e31b0 0x555702d975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555702c24ab0 con 0x5557029ba2f0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555702ca54d0 con 0x5557029c24d0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555702d8e730 con 0x5557029e31b0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b207ee740 10 client.0 _ll_register_callbacks cb 0x555702ce3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51426/0 (socket says 172.21.15.50:51426) 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3320215243 learned_addr learned my addr 172.21.15.50:0/3320215243 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b137fe640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029e31b0 0x555702d975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.081+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d90640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3320215243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029e31b0 msgr2=0x555702d975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029e31b0 0x555702d975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 msgr2=0x555702d90640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d90640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3320215243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555702ca6340 con 0x5557029ba2f0 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b0401b040 tx=0x7f5b04019bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e8890add5a0065 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5af2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.082+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04004230 con 0x5557029ba2f0 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b040043d0 con 0x5557029ba2f0 2024-05-17T04:44:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555702cfac10 con 0x5557029ba2f0 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5b04019e10 con 0x5557029ba2f0 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5b207ee740 10 client.15975 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555702a4e790 con 0x5557029ba2f0 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555702cfac10 con 0x5557029ba2f0 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5b04036020 con 0x5557029ba2f0 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.083+0000 7f5b207ee740 10 client.15975 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b207ee740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b207ee740 20 client.15975 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029a1d80 0x555702d9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b207ee740 4 client.15975 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555702cf43f0 con 0x5557029a1d80 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029a1d80 0x555702d9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5af2ffd640 10 client.15975 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.084+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029a1d80 0x555702d9d740 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbb217f063c6c47 server_cookie=93b0c0e7a3d09c74 in_seq=0 out_seq=0 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.085+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3320215243 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555702cf43f0 con 0x5557029a1d80 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.085+0000 7f5af2ffd640 10 client.15975 handle_command_reply: tid=0 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.085+0000 7f5b207ee740 1 client.15975 shutdown 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 msgr2=0x555702d90100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b0401b040 tx=0x7f5b04019bf0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 shutdown_connections 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029e31b0 0x555702d975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d90640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029a1d80 0x555702d9d740 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3320215243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029ba2f0 0x555702d90100 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 >> 172.21.15.50:0/3320215243 conn(0x5557029d7300 msgr2=0x555702d85010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5af2ffd640 0 client.15975 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 shutdown_connections 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 1 -- 172.21.15.50:0/3320215243 wait complete. 2024-05-17T04:44:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.086+0000 7f5b207ee740 20 client.15975 trim_cache size 0 max 16384 2024-05-17T04:44:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.107+0000 7f5b207ee740 1 Processor -- start 2024-05-17T04:44:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.107+0000 7f5b207ee740 1 -- start start 2024-05-17T04:44:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 0x555702d90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 0x555702d90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 0x555702d90580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:35708/0 (socket says 172.21.15.50:35708) 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d84b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52582/0 (socket says 172.21.15.50:52582) 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b12ffd640 1 -- 172.21.15.50:0/332083189 learned_addr learned my addr 172.21.15.50:0/332083189 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b00012440 con 0x5557029e31b0 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.108+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000fd10 con 0x5557029c24d0 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000db60 con 0x5557029ba2f0 2024-05-17T04:44:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 -- 172.21.15.50:0/332083189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 msgr2=0x555702d90580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 0x555702d90580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 -- 172.21.15.50:0/332083189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 msgr2=0x555702d84b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d84b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.109+0000 7f5b137fe640 1 -- 172.21.15.50:0/332083189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555702ac7620 con 0x5557029e31b0 2024-05-17T04:44:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.110+0000 7f5b137fe640 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d85090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c013010 tx=0x7f5b0c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad52b6d56d8f9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.110+0000 7f5af97fa640 1 -- 172.21.15.50:0/332083189 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c011a50 con 0x5557029e31b0 2024-05-17T04:44:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.111+0000 7f5af97fa640 1 -- 172.21.15.50:0/332083189 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b0c011bf0 con 0x5557029e31b0 2024-05-17T04:44:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.111+0000 7f5af97fa640 1 -- 172.21.15.50:0/332083189 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c01f420 con 0x5557029e31b0 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 msgr2=0x555702d85090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d85090 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c013010 tx=0x7f5b0c01a000 comp rx=0 tx=0).stop 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 shutdown_connections 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d85090 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029c24d0 0x555702d84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 --2- 172.21.15.50:0/332083189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029ba2f0 0x555702d90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 >> 172.21.15.50:0/332083189 conn(0x5557029d7300 msgr2=0x555702d8fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 shutdown_connections 2024-05-17T04:44:55.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.112+0000 7f5b207ee740 1 -- 172.21.15.50:0/332083189 wait complete. 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 -- start start 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 Processor -- start 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d93320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000e1d0 con 0x5557029e31b0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000f640 con 0x5557029ba2f0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000e8a0 con 0x5557029c24d0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.113+0000 7f5b207ee740 10 client.0 _ll_register_callbacks cb 0x7f5b04009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52592/0 (socket says 172.21.15.50:52592) 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b127fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d93320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3239883824 learned_addr learned my addr 172.21.15.50:0/3239883824 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b137fe640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3239883824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 msgr2=0x555702d93320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d93320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3239883824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 msgr2=0x555702d93860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d93860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 -- 172.21.15.50:0/3239883824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555702d93da0 con 0x5557029ba2f0 2024-05-17T04:44:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5b12ffd640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b0402e960 tx=0x7f5b0402ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819bfe75ca4ffef4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5af2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040052e0 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.114+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b04005480 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b0c011bf0 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5b04019c60 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 10 client.25733 fetch_fsmap learned FSMap version 39 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555702e0d230 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b0c011bf0 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5b0403f030 con 0x5557029ba2f0 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 10 client.25733 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:44:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 20 client.25733 populate_metadata read hostname 'smithi050' 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029771c0 0x555702e0fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 4 client.25733 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.115+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555702c6e750 con 0x5557029771c0 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.116+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029771c0 0x555702e0fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.117+0000 7f5b127fc640 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029771c0 0x555702e0fca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae8122f6864e3aa server_cookie=aa06ea02afc1a7d1 in_seq=0 out_seq=0 2024-05-17T04:44:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.117+0000 7f5af2ffd640 10 client.25733 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.130+0000 7f5af2ffd640 1 -- 172.21.15.50:0/3239883824 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555702c6e750 con 0x5557029771c0 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.130+0000 7f5af2ffd640 10 client.25733 handle_command_reply: tid=0 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.130+0000 7f5b207ee740 1 client.25733 shutdown 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 msgr2=0x555702d967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b0402e960 tx=0x7f5b0402ee70 comp rx=0 tx=0).stop 2024-05-17T04:44:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 shutdown_connections 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5557029771c0 0x555702e0fca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557029e31b0 0x555702d93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557029c24d0 0x555702d93320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5af2ffd640 0 client.25733 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 --2- 172.21.15.50:0/3239883824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5557029ba2f0 0x555702d967a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 >> 172.21.15.50:0/3239883824 conn(0x5557029d7300 msgr2=0x555702d85510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 shutdown_connections 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 1 -- 172.21.15.50:0/3239883824 wait complete. 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.131+0000 7f5b207ee740 20 client.25733 trim_cache size 0 max 16384 2024-05-17T04:44:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 msgr2=0x7f5afc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 0x7f5afc0b4d70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x555702d87960 tx=0x7f5b0c009500 comp rx=0 tx=0).stop 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 msgr2=0x7f5b14075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 0x7f5b14075090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005760 tx=0x7f5b08005e70 comp rx=0 tx=0).stop 2024-05-17T04:44:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 shutdown_connections 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5afc0b2860 0x7f5afc0b4d70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b14104bf0 0x7f5b140755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b141042a0 0x7f5b14075090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 --2- 172.21.15.50:0/3817356344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b14103180 0x7f5b140769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 >> 172.21.15.50:0/3817356344 conn(0x7f5b140fe980 msgr2=0x7f5b14107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 shutdown_connections 2024-05-17T04:44:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:44:55.132+0000 7f5b1984e640 1 -- 172.21.15.50:0/3817356344 wait complete. 2024-05-17T04:44:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:56 smithi105 ceph-mon[34795]: pgmap v697: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:56 smithi110 ceph-mon[35497]: pgmap v697: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:56.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:56 smithi050 ceph-mon[31739]: pgmap v697: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 2.6 MiB/s wr, 305 op/s 2024-05-17T04:44:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 360.0 s, 123.8 tps, lat 283.698 ms stddev 98.379 2024-05-17T04:44:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:44:58 smithi105 ceph-mon[34795]: pgmap v698: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-05-17T04:44:58.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:44:58 smithi110 ceph-mon[35497]: pgmap v698: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-05-17T04:44:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:44:58 smithi050 ceph-mon[31739]: pgmap v698: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-05-17T04:45:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:00 smithi105 ceph-mon[34795]: pgmap v699: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-17T04:45:00.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:00 smithi110 ceph-mon[35497]: pgmap v699: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-17T04:45:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:00 smithi050 ceph-mon[31739]: pgmap v699: 129 pgs: 129 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-17T04:45:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:02 smithi105 ceph-mon[34795]: pgmap v700: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-05-17T04:45:02.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:02 smithi110 ceph-mon[35497]: pgmap v700: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-05-17T04:45:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:02 smithi050 ceph-mon[31739]: pgmap v700: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-05-17T04:45:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 365.0 s, 118.0 tps, lat 311.488 ms stddev 140.153 2024-05-17T04:45:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:04 smithi105 ceph-mon[34795]: pgmap v701: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-05-17T04:45:04.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:04 smithi110 ceph-mon[35497]: pgmap v701: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-05-17T04:45:04.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:04 smithi050 ceph-mon[31739]: pgmap v701: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-05-17T04:45:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:06 smithi105 ceph-mon[34795]: pgmap v702: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:45:06.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:06 smithi110 ceph-mon[35497]: pgmap v702: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:45:06.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:06 smithi050 ceph-mon[31739]: pgmap v702: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:45:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 370.0 s, 133.6 tps, lat 276.780 ms stddev 100.555 2024-05-17T04:45:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 -- 172.21.15.50:0/2923635291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c102930 msgr2=0x7f890c102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 --2- 172.21.15.50:0/2923635291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c102930 0x7f890c102d40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009900 tx=0x7f88fc02a410 comp rx=0 tx=0).stop 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 -- 172.21.15.50:0/2923635291 shutdown_connections 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 --2- 172.21.15.50:0/2923635291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c103280 0x7f890c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 --2- 172.21.15.50:0/2923635291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c102930 0x7f890c102d40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 --2- 172.21.15.50:0/2923635291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c101810 0x7f890c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 -- 172.21.15.50:0/2923635291 >> 172.21.15.50:0/2923635291 conn(0x7f890c0fd010 msgr2=0x7f890c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 -- 172.21.15.50:0/2923635291 shutdown_connections 2024-05-17T04:45:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.257+0000 7f891389e640 1 -- 172.21.15.50:0/2923635291 wait complete. 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 Processor -- start 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 -- start start 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101810 0x7f890c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 0x7f890c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10ebd0 con 0x7f890c103280 2024-05-17T04:45:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10ea50 con 0x7f890c102930 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.258+0000 7f891389e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f890c10ed50 con 0x7f890c101810 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8910e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 0x7f890c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f891163a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101810 0x7f890c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8910e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53468/0 (socket says 172.21.15.50:53468) 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8910e39640 1 -- 172.21.15.50:0/1032448954 learned_addr learned my addr 172.21.15.50:0/1032448954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 -- 172.21.15.50:0/1032448954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101810 msgr2=0x7f890c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101810 0x7f890c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 -- 172.21.15.50:0/1032448954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 msgr2=0x7f890c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 -- 172.21.15.50:0/1032448954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1a1350 con 0x7f890c103280 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8910e39640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f8911e3b640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 0x7f890c1a0ce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f89027fc640 1 -- 172.21.15.50:0/1032448954 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890400ce20 con 0x7f890c103280 2024-05-17T04:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f89027fc640 1 -- 172.21.15.50:0/1032448954 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8904005020 con 0x7f890c103280 2024-05-17T04:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.259+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c1a1640 con 0x7f890c103280 2024-05-17T04:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.260+0000 7f89027fc640 1 -- 172.21.15.50:0/1032448954 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904010690 con 0x7f890c103280 2024-05-17T04:45:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.260+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a1be0 con 0x7f890c103280 2024-05-17T04:45:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.263+0000 7f891a839740 1 Processor -- start 2024-05-17T04:45:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.263+0000 7f89027fc640 1 -- 172.21.15.50:0/1032448954 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8904004a40 con 0x7f890c103280 2024-05-17T04:45:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f891a839740 1 -- start start 2024-05-17T04:45:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f891a839740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f891a839740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f891163a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f891163a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53474/0 (socket says 172.21.15.50:53474) 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f8910e39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.264+0000 7f8910e39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:55646/0 (socket says 172.21.15.50:55646) 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891a839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa21dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 -- 172.21.15.50:0/987435061 learned_addr learned my addr 172.21.15.50:0/987435061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556eaa188cc0 con 0x556ea9e524d0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556eaa178860 con 0x556ea9e4a2f0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556eaa17c8a0 con 0x556ea9e731b0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f89027fc640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 0x7f88f80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f8910e39640 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa21dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 -- 172.21.15.50:0/987435061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 msgr2=0x556eaa216770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 -- 172.21.15.50:0/987435061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 msgr2=0x556eaa21dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa21dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 -- 172.21.15.50:0/987435061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaa140d50 con 0x556ea9e4a2f0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f8910e39640 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f89027fc640 1 -- 172.21.15.50:0/1032448954 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f89040e2ba0 con 0x7f890c103280 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f8911e3b640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 0x7f88f80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.265+0000 7f891163a640 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8908009c90 tx=0x7f8908011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc54d140179bf0a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.266+0000 7f88f77fe640 1 -- 172.21.15.50:0/987435061 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890801e070 con 0x556ea9e4a2f0 2024-05-17T04:45:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.266+0000 7f88f77fe640 1 -- 172.21.15.50:0/987435061 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8908019ac0 con 0x556ea9e4a2f0 2024-05-17T04:45:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.266+0000 7f88f77fe640 1 -- 172.21.15.50:0/987435061 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890800e930 con 0x556ea9e4a2f0 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 msgr2=0x556eaa216cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f8911e3b640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 0x7f88f80b4d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f890406a380 tx=0x7f8904077a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8908009c90 tx=0x7f8908011810 comp rx=0 tx=0).stop 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 shutdown_connections 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa21dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa216cf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 --2- 172.21.15.50:0/987435061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa216770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 >> 172.21.15.50:0/987435061 conn(0x556ea9e67300 msgr2=0x556eaa2121a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 shutdown_connections 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- 172.21.15.50:0/987435061 wait complete. 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 -- start start 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.267+0000 7f891a839740 1 Processor -- start 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 0x556eaa220060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 0x556eaa2205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa227500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556eaa0b4ab0 con 0x556ea9e4a2f0 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556eaa1354d0 con 0x556ea9e524d0 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556eaa21e690 con 0x556ea9e731b0 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891a839740 10 client.0 _ll_register_callbacks cb 0x556eaa173950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f8910e39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 0x556eaa220060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f8910e39640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 0x556eaa220060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48136/0 (socket says 172.21.15.50:48136) 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f8910e39640 1 -- 172.21.15.50:0/3416019640 learned_addr learned my addr 172.21.15.50:0/3416019640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f891163a640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 0x556eaa2205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.268+0000 7f8910e39640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa227500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 -- 172.21.15.50:0/3416019640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 msgr2=0x556eaa227500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa227500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 -- 172.21.15.50:0/3416019640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 msgr2=0x556eaa220060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 0x556eaa220060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 -- 172.21.15.50:0/3416019640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa136340 con 0x556ea9e524d0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891163a640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 0x556eaa2205a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f890801c040 tx=0x7f89080199b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bf3688822eeb19 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f88f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890801e050 con 0x556ea9e524d0 2024-05-17T04:45:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8908019ac0 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8908005450 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8908032050 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891a839740 10 client.25745 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.269+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556ea9ede790 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8908019ac0 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f890802d030 con 0x556ea9e524d0 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 10 client.25745 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 20 client.25745 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e31d80 0x556eaa22ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 4 client.25745 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:45:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaa18ac10 con 0x556ea9e31d80 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.270+0000 7f8910e39640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e31d80 0x556eaa22ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.271+0000 7f88f4ff9640 10 client.25745 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.271+0000 7f8910e39640 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e31d80 0x556eaa22ae10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7b04eb2f165aa1 server_cookie=3768846a800407 in_seq=0 out_seq=0 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.272+0000 7f88f4ff9640 1 -- 172.21.15.50:0/3416019640 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556eaa18ac10 con 0x556ea9e31d80 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.272+0000 7f88f4ff9640 10 client.25745 handle_command_reply: tid=0 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.272+0000 7f891a839740 1 client.25745 shutdown 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.272+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 msgr2=0x556eaa2205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.272+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 0x556eaa2205a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f890801c040 tx=0x7f89080199b0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 shutdown_connections 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e731b0 0x556eaa227500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e524d0 0x556eaa2205a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e31d80 0x556eaa22ae10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 --2- 172.21.15.50:0/3416019640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e4a2f0 0x556eaa220060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 >> 172.21.15.50:0/3416019640 conn(0x556ea9e67300 msgr2=0x556eaa215010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f88f4ff9640 0 client.25745 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 shutdown_connections 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 1 -- 172.21.15.50:0/3416019640 wait complete. 2024-05-17T04:45:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.273+0000 7f891a839740 20 client.25745 trim_cache size 0 max 16384 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 Processor -- start 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 -- start start 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 0x556eaa219e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e731b0 0x556eaa21a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ac660 con 0x556ea9e731b0 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9f30 con 0x556ea9e4a2f0 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891a839740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a7d80 con 0x556ea9e524d0 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f8910e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f8910e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53492/0 (socket says 172.21.15.50:53492) 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f8910e39640 1 -- 172.21.15.50:0/3568943532 learned_addr learned my addr 172.21.15.50:0/3568943532 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f891163a640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 0x556eaa219e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.293+0000 7f8910e39640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e731b0 0x556eaa21a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 -- 172.21.15.50:0/3568943532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 msgr2=0x556eaa219e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 0x556eaa219e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 -- 172.21.15.50:0/3568943532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e731b0 msgr2=0x556eaa21a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e731b0 0x556eaa21a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 -- 172.21.15.50:0/3568943532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea9f57620 con 0x556ea9e4a2f0 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f891163a640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 0x556eaa219e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f8910e39640 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006010 tx=0x7f88fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b858d79cc01b7be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f88f6ffd640 1 -- 172.21.15.50:0/3568943532 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc035040 con 0x556ea9e4a2f0 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f88f6ffd640 1 -- 172.21.15.50:0/3568943532 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88fc031990 con 0x556ea9e4a2f0 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.294+0000 7f88f6ffd640 1 -- 172.21.15.50:0/3568943532 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc056040 con 0x556ea9e4a2f0 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 -- 172.21.15.50:0/3568943532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 msgr2=0x556eaa219900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006010 tx=0x7f88fc009260 comp rx=0 tx=0).stop 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 -- 172.21.15.50:0/3568943532 shutdown_connections 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e731b0 0x556eaa21a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e524d0 0x556eaa219e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 --2- 172.21.15.50:0/3568943532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e4a2f0 0x556eaa219900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 -- 172.21.15.50:0/3568943532 >> 172.21.15.50:0/3568943532 conn(0x556ea9e67300 msgr2=0x556eaa215790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 -- 172.21.15.50:0/3568943532 shutdown_connections 2024-05-17T04:45:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.295+0000 7f891a839740 1 -- 172.21.15.50:0/3568943532 wait complete. 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.296+0000 7f891a839740 1 -- start start 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.296+0000 7f891a839740 1 Processor -- start 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 0x556eaa28c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa28cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e731b0 0x556eaa293650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a83f0 con 0x556ea9e524d0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9860 con 0x556ea9e731b0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a8ac0 con 0x556ea9e4a2f0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 0x556eaa28c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891163a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa28cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891163a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa28cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48154/0 (socket says 172.21.15.50:48154) 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891163a640 1 -- 172.21.15.50:0/222270200 learned_addr learned my addr 172.21.15.50:0/222270200 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f891a839740 10 client.0 _ll_register_callbacks cb 0x7f890801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e731b0 0x556eaa293650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 -- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e731b0 msgr2=0x556eaa293650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e731b0 0x556eaa293650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 -- 172.21.15.50:0/222270200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 msgr2=0x556eaa28cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa28cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.297+0000 7f8910e39640 1 -- 172.21.15.50:0/222270200 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa293c40 con 0x556ea9e4a2f0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f8910e39640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 0x556eaa28c860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88fc02a9d0 tx=0x7f88fc061040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb153d41727f0686 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc035070 con 0x556ea9e4a2f0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f88fc031b40 con 0x556ea9e4a2f0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc031990 con 0x556ea9e4a2f0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f88fc031d40 con 0x556ea9e4a2f0 2024-05-17T04:45:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 10 client.25735 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556eaa294e70 con 0x556ea9e4a2f0 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88fc03b9a0 con 0x556ea9e4a2f0 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc05c040 con 0x556ea9e4a2f0 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 10 client.25735 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 10 client.25735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 20 client.25735 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.298+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e071c0 0x556eaa297830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.299+0000 7f891a839740 4 client.25735 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.299+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f88fc031990 con 0x556ea9e071c0 2024-05-17T04:45:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.299+0000 7f891163a640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e071c0 0x556eaa297830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.299+0000 7f88f4ff9640 10 client.25735 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.300+0000 7f891163a640 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e071c0 0x556eaa297830 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee9e676c08cfbc4 server_cookie=d011fc2f59429a26 in_seq=0 out_seq=0 2024-05-17T04:45:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:08 smithi105 ceph-mon[34795]: pgmap v703: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-17T04:45:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f88f4ff9640 1 -- 172.21.15.50:0/222270200 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f88fc031990 con 0x556ea9e071c0 2024-05-17T04:45:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f88f4ff9640 10 client.25735 handle_command_reply: tid=0 2024-05-17T04:45:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 client.25735 shutdown 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 msgr2=0x556eaa28c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 0x556eaa28c860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88fc02a9d0 tx=0x7f88fc061040 comp rx=0 tx=0).stop 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 shutdown_connections 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ea9e731b0 0x556eaa293650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556ea9e071c0 0x556eaa297830 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ea9e524d0 0x556eaa28cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 --2- 172.21.15.50:0/222270200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ea9e4a2f0 0x556eaa28c860 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f88f4ff9640 0 client.25735 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 >> 172.21.15.50:0/222270200 conn(0x556ea9e67300 msgr2=0x556eaa215560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 shutdown_connections 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 1 -- 172.21.15.50:0/222270200 wait complete. 2024-05-17T04:45:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.315+0000 7f891a839740 20 client.25735 trim_cache size 0 max 16384 2024-05-17T04:45:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:45:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:45:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:45:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 msgr2=0x7f88f80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 0x7f88f80b4d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f890406a380 tx=0x7f8904077a50 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 msgr2=0x7f890c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 0x7f890c1a0ce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8904009fd0 tx=0x7f890400ef10 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 shutdown_connections 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f88f80b2860 0x7f88f80b4d70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f890c103280 0x7f890c1a0ce0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f890c102930 0x7f890c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 --2- 172.21.15.50:0/1032448954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101810 0x7f890c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 >> 172.21.15.50:0/1032448954 conn(0x7f890c0fd010 msgr2=0x7f890c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 shutdown_connections 2024-05-17T04:45:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:08.316+0000 7f891389e640 1 -- 172.21.15.50:0/1032448954 wait complete. 2024-05-17T04:45:08.320 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:08 smithi110 ceph-mon[35497]: pgmap v703: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-17T04:45:08.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:08 smithi050 ceph-mon[31739]: pgmap v703: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 289 op/s 2024-05-17T04:45:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:10 smithi105 ceph-mon[34795]: pgmap v704: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-17T04:45:10.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:10 smithi110 ceph-mon[35497]: pgmap v704: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-17T04:45:10.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:10 smithi050 ceph-mon[31739]: pgmap v704: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-17T04:45:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:12 smithi105 ceph-mon[34795]: pgmap v705: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-17T04:45:12.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:12 smithi110 ceph-mon[35497]: pgmap v705: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-17T04:45:12.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:12 smithi050 ceph-mon[31739]: pgmap v705: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.6 MiB/s wr, 310 op/s 2024-05-17T04:45:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 375.0 s, 123.6 tps, lat 285.816 ms stddev 76.476 2024-05-17T04:45:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:14 smithi105 ceph-mon[34795]: pgmap v706: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-17T04:45:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:14 smithi110 ceph-mon[35497]: pgmap v706: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-17T04:45:14.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:14 smithi050 ceph-mon[31739]: pgmap v706: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-05-17T04:45:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:16 smithi105 ceph-mon[34795]: pgmap v707: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-17T04:45:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:16 smithi110 ceph-mon[35497]: pgmap v707: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-17T04:45:16.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:16 smithi050 ceph-mon[31739]: pgmap v707: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-17T04:45:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 380.0 s, 108.6 tps, lat 333.817 ms stddev 99.068 2024-05-17T04:45:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:18 smithi105 ceph-mon[34795]: pgmap v708: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.4 MiB/s wr, 312 op/s 2024-05-17T04:45:18.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:18 smithi110 ceph-mon[35497]: pgmap v708: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.4 MiB/s wr, 312 op/s 2024-05-17T04:45:18.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:18 smithi050 ceph-mon[31739]: pgmap v708: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.4 MiB/s wr, 312 op/s 2024-05-17T04:45:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:20 smithi105 ceph-mon[34795]: pgmap v709: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:45:20.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:20 smithi110 ceph-mon[35497]: pgmap v709: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:45:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:20 smithi050 ceph-mon[31739]: pgmap v709: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.447+0000 7f78f04b9640 1 -- 172.21.15.50:0/192212577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8101a60 msgr2=0x7f78e8103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.447+0000 7f78f04b9640 1 --2- 172.21.15.50:0/192212577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8101a60 0x7f78e8103e80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009860 tx=0x7f78e402a340 comp rx=0 tx=0).stop 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 -- 172.21.15.50:0/192212577 shutdown_connections 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 --2- 172.21.15.50:0/192212577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8106d20 0x7f78e8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 --2- 172.21.15.50:0/192212577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 --2- 172.21.15.50:0/192212577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8101a60 0x7f78e8103e80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 -- 172.21.15.50:0/192212577 >> 172.21.15.50:0/192212577 conn(0x7f78e80fd880 msgr2=0x7f78e80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 -- 172.21.15.50:0/192212577 shutdown_connections 2024-05-17T04:45:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.448+0000 7f78f04b9640 1 -- 172.21.15.50:0/192212577 wait complete. 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 Processor -- start 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 -- start start 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 0x7f78e819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8106d20 0x7f78e81a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8115140 con 0x7f78e81043c0 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8114fc0 con 0x7f78e8101a60 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78f04b9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81152c0 con 0x7f78e8106d20 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78eda54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 0x7f78e819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78eda54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35374/0 (socket says 172.21.15.50:35374) 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78eda54640 1 -- 172.21.15.50:0/1423377913 learned_addr learned my addr 172.21.15.50:0/1423377913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78eea56640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8106d20 0x7f78e81a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 -- 172.21.15.50:0/1423377913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8106d20 msgr2=0x7f78e81a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8106d20 0x7f78e81a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 -- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 msgr2=0x7f78e81a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.449+0000 7f78ee255640 1 -- 172.21.15.50:0/1423377913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e81a7950 con 0x7f78e8101a60 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78ee255640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 0x7f78e819fe10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78e4035040 tx=0x7f78e402ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78df7fe640 1 -- 172.21.15.50:0/1423377913 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4040020 con 0x7f78e8101a60 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e81a7be0 con 0x7f78e8101a60 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78df7fe640 1 -- 172.21.15.50:0/1423377913 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78e40307f0 con 0x7f78e8101a60 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78df7fe640 1 -- 172.21.15.50:0/1423377913 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4037040 con 0x7f78e8101a60 2024-05-17T04:45:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.450+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a80f0 con 0x7f78e8101a60 2024-05-17T04:45:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.453+0000 7f78f7370740 1 Processor -- start 2024-05-17T04:45:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.453+0000 7f78f7370740 1 -- start start 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ad9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 0x560766ae0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560766a4bcc0 con 0x56076670d2f0 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560766a3b860 con 0x5607667154d0 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78f7370740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560766a3f8a0 con 0x5607667361b0 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78df7fe640 1 -- 172.21.15.50:0/1423377913 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f78e4043020 con 0x7f78e8101a60 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59768/0 (socket says 172.21.15.50:59768) 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 -- 172.21.15.50:0/2388728978 learned_addr learned my addr 172.21.15.50:0/2388728978 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eda54640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 0x560766ae0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78ee255640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ad9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 -- 172.21.15.50:0/2388728978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 msgr2=0x560766ae0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 0x560766ae0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 -- 172.21.15.50:0/2388728978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 msgr2=0x560766ad9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ad9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 -- 172.21.15.50:0/2388728978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560766a03d50 con 0x5607667361b0 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eda54640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 0x560766ae0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78ee255640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ad9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.454+0000 7f78eea56640 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e580 tx=0x7f78e00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910fcd3884979c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78dcbfe640 1 -- 172.21.15.50:0/2388728978 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003970 con 0x5607667361b0 2024-05-17T04:45:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78dcbfe640 1 -- 172.21.15.50:0/2388728978 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78e000d830 con 0x5607667361b0 2024-05-17T04:45:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78dcbfe640 1 -- 172.21.15.50:0/2388728978 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0004450 con 0x5607667361b0 2024-05-17T04:45:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78df7fe640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 0x7f78d40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78ee255640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 0x7f78d40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.455+0000 7f78df7fe640 1 -- 172.21.15.50:0/1423377913 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f78e4102550 con 0x7f78e8101a60 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 -- 172.21.15.50:0/2388728978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 msgr2=0x560766ad9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78ee255640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 0x7f78d40b4d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x560766adab70 tx=0x7f78e40d0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e580 tx=0x7f78e00095c0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 -- 172.21.15.50:0/2388728978 shutdown_connections 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667154d0 0x560766ae0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ad9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 --2- 172.21.15.50:0/2388728978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667361b0 0x560766ad9770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 -- 172.21.15.50:0/2388728978 >> 172.21.15.50:0/2388728978 conn(0x56076672a300 msgr2=0x560766ad51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 -- 172.21.15.50:0/2388728978 shutdown_connections 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.456+0000 7f78f7370740 1 -- 172.21.15.50:0/2388728978 wait complete. 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 -- start start 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 Processor -- start 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ae30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667361b0 0x560766aea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560766977ab0 con 0x56076670d2f0 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5607669f84d0 con 0x5607667361b0 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560766ae1690 con 0x5607667154d0 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78f7370740 10 client.0 _ll_register_callbacks cb 0x560766a36950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78eea56640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ae30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78eda54640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78eda54640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59784/0 (socket says 172.21.15.50:59784) 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78eda54640 1 -- 172.21.15.50:0/2015873511 learned_addr learned my addr 172.21.15.50:0/2015873511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.457+0000 7f78eea56640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ae30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35400/0 (socket says 172.21.15.50:35400) 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eea56640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667361b0 0x560766aea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 -- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667361b0 msgr2=0x560766aea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667361b0 0x560766aea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 -- 172.21.15.50:0/2015873511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 msgr2=0x560766ae30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ae30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 -- 172.21.15.50:0/2015873511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607669f9340 con 0x5607667154d0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78eda54640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f78d8012750 tx=0x7f78d8012c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85ed38111a794b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8012e60 con 0x5607667154d0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f78d8004050 con 0x5607667154d0 2024-05-17T04:45:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.458+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e000d830 con 0x5607667154d0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78d8025050 con 0x5607667154d0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 10 client.25747 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5607667a1790 con 0x5607667154d0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78e000d830 con 0x5607667154d0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f78d80037b0 con 0x5607667154d0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 10 client.25747 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 20 client.25747 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607666f4d80 0x560766af09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 4 client.25747 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560766a4dc10 con 0x5607666f4d80 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.459+0000 7f78eea56640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607666f4d80 0x560766af09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.460+0000 7f78d1ffb640 10 client.25747 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.460+0000 7f78eea56640 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607666f4d80 0x560766af09f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0723d6ad67bd80 server_cookie=cb719e51f2df1ae0 in_seq=0 out_seq=0 2024-05-17T04:45:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78d1ffb640 1 -- 172.21.15.50:0/2015873511 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560766a4dc10 con 0x5607666f4d80 2024-05-17T04:45:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78d1ffb640 10 client.25747 handle_command_reply: tid=0 2024-05-17T04:45:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 client.25747 shutdown 2024-05-17T04:45:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 msgr2=0x560766ae3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f78d8012750 tx=0x7f78d8012c10 comp rx=0 tx=0).stop 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 shutdown_connections 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607667361b0 0x560766aea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766ae3610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607666f4d80 0x560766af09f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78d1ffb640 0 client.25747 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 --2- 172.21.15.50:0/2015873511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766ae30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.461+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 >> 172.21.15.50:0/2015873511 conn(0x56076672a300 msgr2=0x560766ad41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.462+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 shutdown_connections 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.462+0000 7f78f7370740 1 -- 172.21.15.50:0/2015873511 wait complete. 2024-05-17T04:45:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.462+0000 7f78f7370740 20 client.25747 trim_cache size 0 max 16384 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 Processor -- start 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 -- start start 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607666f4d80 0x560766ad5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766aea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b4de0 con 0x56076670d2f0 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b26b0 con 0x5607666f4d80 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78f7370740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b0500 con 0x5607667154d0 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78eda54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78eda54640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35402/0 (socket says 172.21.15.50:35402) 2024-05-17T04:45:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78eda54640 1 -- 172.21.15.50:0/3788169604 learned_addr learned my addr 172.21.15.50:0/3788169604 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78eea56640 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607666f4d80 0x560766ad5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.483+0000 7f78eea56640 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766aea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 -- 172.21.15.50:0/3788169604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 msgr2=0x560766aea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766aea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 -- 172.21.15.50:0/3788169604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607666f4d80 msgr2=0x560766ad5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607666f4d80 0x560766ad5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 -- 172.21.15.50:0/3788169604 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076681a620 con 0x56076670d2f0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78eda54640 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f78d8012420 tx=0x7f78d800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e920b34122cab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78d3fff640 1 -- 172.21.15.50:0/3788169604 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80035f0 con 0x56076670d2f0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78d3fff640 1 -- 172.21.15.50:0/3788169604 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d800fdb0 con 0x56076670d2f0 2024-05-17T04:45:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.484+0000 7f78d3fff640 1 -- 172.21.15.50:0/3788169604 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80054f0 con 0x56076670d2f0 2024-05-17T04:45:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 -- 172.21.15.50:0/3788169604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 msgr2=0x560766aea490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f78d8012420 tx=0x7f78d800f810 comp rx=0 tx=0).stop 2024-05-17T04:45:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 -- 172.21.15.50:0/3788169604 shutdown_connections 2024-05-17T04:45:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607667154d0 0x560766aea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56076670d2f0 0x560766aea490 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 --2- 172.21.15.50:0/3788169604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607666f4d80 0x560766ad5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 -- 172.21.15.50:0/3788169604 >> 172.21.15.50:0/3788169604 conn(0x56076672a300 msgr2=0x560766ad41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 -- 172.21.15.50:0/3788169604 shutdown_connections 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.485+0000 7f78f7370740 1 -- 172.21.15.50:0/3788169604 wait complete. 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 -- start start 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 Processor -- start 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 0x560766ae6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607667154d0 0x560766ae96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b0b70 con 0x5607667154d0 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b1fe0 con 0x56076670d2f0 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81b1240 con 0x5607666f4d80 2024-05-17T04:45:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78f7370740 10 client.0 _ll_register_callbacks cb 0x7f78d8009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eea56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eea56640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607667154d0 0x560766ae96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eda54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 0x560766ae6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eea56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:59802/0 (socket says 172.21.15.50:59802) 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eea56640 1 -- 172.21.15.50:0/1917779062 learned_addr learned my addr 172.21.15.50:0/1917779062 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.486+0000 7f78eda54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 0x560766ae6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56934/0 (socket says 172.21.15.50:56934) 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 -- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 msgr2=0x560766ae6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 0x560766ae6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 -- 172.21.15.50:0/1917779062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607667154d0 msgr2=0x560766ae96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607667154d0 0x560766ae96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 -- 172.21.15.50:0/1917779062 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560766ae9c90 con 0x5607666f4d80 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78eea56640 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78e000ff60 tx=0x7f78e0026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e32a3006441f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:45:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003970 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f78e001c040 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d800fdb0 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78e0003b10 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78f7370740 10 client.25753 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.487+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560766aeafb0 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d800fdb0 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f78e000b040 con 0x5607666f4d80 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 10 client.25753 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 20 client.25753 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607667361b0 0x560766b5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 4 client.25753 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:45:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78eda54640 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607667361b0 0x560766b5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.488+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560766a473f0 con 0x5607667361b0 2024-05-17T04:45:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.490+0000 7f78eda54640 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607667361b0 0x560766b5b280 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a8fc2bf9e6c709 server_cookie=f39bd1a42755516c in_seq=0 out_seq=0 2024-05-17T04:45:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.490+0000 7f78d1ffb640 10 client.25753 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.519+0000 7f78d1ffb640 1 -- 172.21.15.50:0/1917779062 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560766a473f0 con 0x5607667361b0 2024-05-17T04:45:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.519+0000 7f78d1ffb640 10 client.25753 handle_command_reply: tid=0 2024-05-17T04:45:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.519+0000 7f78f7370740 1 client.25753 shutdown 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 msgr2=0x560766ae6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78e000ff60 tx=0x7f78e0026040 comp rx=0 tx=0).stop 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 shutdown_connections 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607667361b0 0x560766b5b280 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607667154d0 0x560766ae96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78d1ffb640 0 client.25753 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56076670d2f0 0x560766ae6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 --2- 172.21.15.50:0/1917779062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607666f4d80 0x560766ae6220 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 >> 172.21.15.50:0/1917779062 conn(0x56076672a300 msgr2=0x560766ad41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 shutdown_connections 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 1 -- 172.21.15.50:0/1917779062 wait complete. 2024-05-17T04:45:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.520+0000 7f78f7370740 20 client.25753 trim_cache size 0 max 16384 2024-05-17T04:45:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:45:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:45:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.521+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 msgr2=0x7f78d40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 0x7f78d40b4d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x560766adab70 tx=0x7f78e40d0040 comp rx=0 tx=0).stop 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 msgr2=0x7f78e819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 0x7f78e819fe10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78e4035040 tx=0x7f78e402ad70 comp rx=0 tx=0).stop 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 shutdown_connections 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78e8106d20 0x7f78e81a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78d40b2860 0x7f78d40b4d70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78e81043c0 0x7f78e81a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 --2- 172.21.15.50:0/1423377913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8101a60 0x7f78e819fe10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 >> 172.21.15.50:0/1423377913 conn(0x7f78e80fd880 msgr2=0x7f78e8102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 shutdown_connections 2024-05-17T04:45:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:21.522+0000 7f78f04b9640 1 -- 172.21.15.50:0/1423377913 wait complete. 2024-05-17T04:45:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:22 smithi105 ceph-mon[34795]: pgmap v710: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 297 op/s 2024-05-17T04:45:22.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:22 smithi110 ceph-mon[35497]: pgmap v710: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 297 op/s 2024-05-17T04:45:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:22 smithi050 ceph-mon[31739]: pgmap v710: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.5 MiB/s wr, 297 op/s 2024-05-17T04:45:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 385.0 s, 111.2 tps, lat 322.521 ms stddev 87.533 2024-05-17T04:45:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:24 smithi105 ceph-mon[34795]: pgmap v711: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-05-17T04:45:24.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:24 smithi110 ceph-mon[35497]: pgmap v711: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-05-17T04:45:24.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:24 smithi050 ceph-mon[31739]: pgmap v711: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 2.2 MiB/s wr, 304 op/s 2024-05-17T04:45:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:26 smithi105 ceph-mon[34795]: pgmap v712: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 306 op/s 2024-05-17T04:45:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:26 smithi110 ceph-mon[35497]: pgmap v712: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 306 op/s 2024-05-17T04:45:26.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:26 smithi050 ceph-mon[31739]: pgmap v712: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.6 MiB/s wr, 306 op/s 2024-05-17T04:45:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 390.0 s, 129.6 tps, lat 276.728 ms stddev 101.884 2024-05-17T04:45:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:28 smithi105 ceph-mon[34795]: pgmap v713: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-17T04:45:28.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:28 smithi110 ceph-mon[35497]: pgmap v713: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-17T04:45:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:28 smithi050 ceph-mon[31739]: pgmap v713: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-17T04:45:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:30 smithi105 ceph-mon[34795]: pgmap v714: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-17T04:45:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:30 smithi110 ceph-mon[35497]: pgmap v714: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-17T04:45:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:30 smithi050 ceph-mon[31739]: pgmap v714: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-17T04:45:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:32 smithi105 ceph-mon[34795]: pgmap v715: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-05-17T04:45:32.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:32 smithi110 ceph-mon[35497]: pgmap v715: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-05-17T04:45:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:32 smithi050 ceph-mon[31739]: pgmap v715: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-05-17T04:45:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 395.0 s, 128.2 tps, lat 280.528 ms stddev 94.763 2024-05-17T04:45:34.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:34 smithi105 ceph-mon[34795]: pgmap v716: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-05-17T04:45:34.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:34 smithi110 ceph-mon[35497]: pgmap v716: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-05-17T04:45:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:34 smithi050 ceph-mon[31739]: pgmap v716: 129 pgs: 129 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-05-17T04:45:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.668+0000 7fb377fff640 1 -- 172.21.15.50:0/3184954863 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3700307b0 con 0x7fb380101170 2024-05-17T04:45:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.669+0000 7fb38748f640 1 -- 172.21.15.50:0/3184954863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 msgr2=0x7fb380101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.669+0000 7fb38748f640 1 --2- 172.21.15.50:0/3184954863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb380101580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb370009890 tx=0x7fb37002a360 comp rx=0 tx=0).stop 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 -- 172.21.15.50:0/3184954863 shutdown_connections 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 --2- 172.21.15.50:0/3184954863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380101ac0 0x7fb38010b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 --2- 172.21.15.50:0/3184954863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb380101580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 --2- 172.21.15.50:0/3184954863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380104730 0x7fb380104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 -- 172.21.15.50:0/3184954863 >> 172.21.15.50:0/3184954863 conn(0x7fb3800fd010 msgr2=0x7fb3800ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 -- 172.21.15.50:0/3184954863 shutdown_connections 2024-05-17T04:45:34.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.672+0000 7fb38748f640 1 -- 172.21.15.50:0/3184954863 wait complete. 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 Processor -- start 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 -- start start 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb38019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380101ac0 0x7fb38019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 0x7fb3801a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010eb70 con 0x7fb380104730 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010e9f0 con 0x7fb380101ac0 2024-05-17T04:45:34.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.673+0000 7fb38748f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb38010ecf0 con 0x7fb380101170 2024-05-17T04:45:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 0x7fb3801a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 0x7fb3801a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52714/0 (socket says 172.21.15.50:52714) 2024-05-17T04:45:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb385a2c640 1 -- 172.21.15.50:0/3984180029 learned_addr learned my addr 172.21.15.50:0/3984180029 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb38019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb384a2a640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380101ac0 0x7fb38019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 -- 172.21.15.50:0/3984180029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380101ac0 msgr2=0x7fb38019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380101ac0 0x7fb38019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 -- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 msgr2=0x7fb3801a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 0x7fb3801a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.674+0000 7fb38522b640 1 -- 172.21.15.50:0/3984180029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3801a3570 con 0x7fb380101170 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb38522b640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb38019ba30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005fd0 tx=0x7fb37c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb3767fc640 1 -- 172.21.15.50:0/3984180029 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00ccc0 con 0x7fb380101170 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3801a3860 con 0x7fb380101170 2024-05-17T04:45:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3801a3dd0 con 0x7fb380101170 2024-05-17T04:45:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb3767fc640 1 -- 172.21.15.50:0/3984180029 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb37c00ee70 con 0x7fb380101170 2024-05-17T04:45:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.675+0000 7fb3767fc640 1 -- 172.21.15.50:0/3984180029 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c018680 con 0x7fb380101170 2024-05-17T04:45:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.679+0000 7fb38e427740 1 Processor -- start 2024-05-17T04:45:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.679+0000 7fb38e427740 1 -- start start 2024-05-17T04:45:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558f7a023cc0 con 0x558f79ce52f0 2024-05-17T04:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f7a013860 con 0x558f79ced4d0 2024-05-17T04:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb38e427740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f7a0178a0 con 0x558f79d0e1b0 2024-05-17T04:45:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:40862/0 (socket says 172.21.15.50:40862) 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.680+0000 7fb385a2c640 1 -- 172.21.15.50:0/100369710 learned_addr learned my addr 172.21.15.50:0/100369710 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb384a2a640 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 -- 172.21.15.50:0/100369710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 msgr2=0x558f7a0b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 -- 172.21.15.50:0/100369710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 msgr2=0x558f7a0b1cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0b1cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 -- 172.21.15.50:0/100369710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f79fdbd50 con 0x558f79d0e1b0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb385a2c640 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3780114c0 tx=0x7fb378011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891cb40cd270c94c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb36b7fe640 1 -- 172.21.15.50:0/100369710 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378004b20 con 0x558f79d0e1b0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb36b7fe640 1 -- 172.21.15.50:0/100369710 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb378004cc0 con 0x558f79d0e1b0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb36b7fe640 1 -- 172.21.15.50:0/100369710 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378005420 con 0x558f79d0e1b0 2024-05-17T04:45:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.681+0000 7fb3767fc640 1 -- 172.21.15.50:0/3984180029 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb37c005030 con 0x7fb380101170 2024-05-17T04:45:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 -- 172.21.15.50:0/100369710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 msgr2=0x558f7a0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3780114c0 tx=0x7fb378011980 comp rx=0 tx=0).stop 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 -- 172.21.15.50:0/100369710 shutdown_connections 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 --2- 172.21.15.50:0/100369710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b1770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 -- 172.21.15.50:0/100369710 >> 172.21.15.50:0/100369710 conn(0x558f79d02300 msgr2=0x558f7a0ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 -- 172.21.15.50:0/100369710 shutdown_connections 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb38e427740 1 -- 172.21.15.50:0/100369710 wait complete. 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb3767fc640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 0x7fb36c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb384a2a640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 0x7fb36c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.683+0000 7fb3767fc640 1 -- 172.21.15.50:0/3984180029 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb37c0e2700 con 0x7fb380101170 2024-05-17T04:45:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.684+0000 7fb38e427740 1 -- start start 2024-05-17T04:45:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.684+0000 7fb38e427740 1 Processor -- start 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb384a2a640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 0x7fb36c0b4c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a950 tx=0x7fb3700058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 0x558f7a0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558f79f4fab0 con 0x558f79ced4d0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f79fd04d0 con 0x558f79d0e1b0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f7a0b96e0 con 0x558f79ce52f0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38e427740 10 client.0 _ll_register_callbacks cb 0x558f7a00e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 0x558f7a0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb385a2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 0x558f7a0cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:33728/0 (socket says 172.21.15.50:33728) 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:40874/0 (socket says 172.21.15.50:40874) 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb385a2c640 1 -- 172.21.15.50:0/1972576793 learned_addr learned my addr 172.21.15.50:0/1972576793 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb384a2a640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 -- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 msgr2=0x558f7a0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 0x558f7a0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 -- 172.21.15.50:0/1972576793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 msgr2=0x558f7a0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.685+0000 7fb38522b640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb38522b640 1 -- 172.21.15.50:0/1972576793 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f79fd1340 con 0x558f79ce52f0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb38522b640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558f7a0b2b70 tx=0x7fb37c0a5180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec47939aed3cc04e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0adce0 con 0x558f79ce52f0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb378004cc0 con 0x558f79ce52f0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb37c0a9b90 con 0x558f79ce52f0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb37c0b2790 con 0x558f79ce52f0 2024-05-17T04:45:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb38e427740 10 client.25765 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558f79d79790 con 0x558f79ce52f0 2024-05-17T04:45:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.686+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb378004cc0 con 0x558f79ce52f0 2024-05-17T04:45:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb37c0a8b90 con 0x558f79ce52f0 2024-05-17T04:45:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 10 client.25765 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 20 client.25765 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79cccd80 0x558f7a0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 4 client.25765 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.687+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f7a025c10 con 0x558f79cccd80 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.688+0000 7fb384a2a640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79cccd80 0x558f7a0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.688+0000 7fb384a2a640 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79cccd80 0x558f7a0d14d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a7deb2677f1569 server_cookie=982f89a5512fe1fd in_seq=0 out_seq=0 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.688+0000 7fb368ff9640 10 client.25765 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb368ff9640 1 -- 172.21.15.50:0/1972576793 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f7a025c10 con 0x558f79cccd80 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb368ff9640 10 client.25765 handle_command_reply: tid=0 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 client.25765 shutdown 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 msgr2=0x558f7a0c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x558f7a0b2b70 tx=0x7fb37c0a5180 comp rx=0 tx=0).stop 2024-05-17T04:45:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 shutdown_connections 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79d0e1b0 0x558f7a0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79cccd80 0x558f7a0d14d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 --2- 172.21.15.50:0/1972576793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79ce52f0 0x558f7a0c3830 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 >> 172.21.15.50:0/1972576793 conn(0x558f79d02300 msgr2=0x558f7a0ac1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.689+0000 7fb368ff9640 0 client.25765 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.690+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 shutdown_connections 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.690+0000 7fb38e427740 1 -- 172.21.15.50:0/1972576793 wait complete. 2024-05-17T04:45:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.690+0000 7fb38e427740 20 client.25765 trim_cache size 0 max 16384 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.712+0000 7fb38e427740 1 Processor -- start 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.712+0000 7fb38e427740 1 -- start start 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640123e0 con 0x558f79ced4d0 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400fcb0 con 0x558f79ce52f0 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38e427740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400db00 con 0x558f79d0e1b0 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:33738/0 (socket says 172.21.15.50:33738) 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 -- 172.21.15.50:0/1445593963 learned_addr learned my addr 172.21.15.50:0/1445593963 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb384a2a640 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb385a2c640 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 -- 172.21.15.50:0/1445593963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 msgr2=0x558f7a0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 -- 172.21.15.50:0/1445593963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 msgr2=0x558f7a0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.713+0000 7fb38522b640 1 -- 172.21.15.50:0/1445593963 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f79df2620 con 0x558f79ce52f0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.714+0000 7fb38522b640 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0a6ba0 tx=0x7fb37c0e2600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e7f6f9aa733e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.714+0000 7fb36affd640 1 -- 172.21.15.50:0/1445593963 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0a8b90 con 0x558f79ce52f0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.714+0000 7fb36affd640 1 -- 172.21.15.50:0/1445593963 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb37c0a52a0 con 0x558f79ce52f0 2024-05-17T04:45:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.714+0000 7fb36affd640 1 -- 172.21.15.50:0/1445593963 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0a9df0 con 0x558f79ce52f0 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 -- 172.21.15.50:0/1445593963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 msgr2=0x558f7a0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0a6ba0 tx=0x7fb37c0e2600 comp rx=0 tx=0).stop 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 -- 172.21.15.50:0/1445593963 shutdown_connections 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ced4d0 0x558f7a0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 --2- 172.21.15.50:0/1445593963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ce52f0 0x558f7a0ad200 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 -- 172.21.15.50:0/1445593963 >> 172.21.15.50:0/1445593963 conn(0x558f79d02300 msgr2=0x558f7a0acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 -- 172.21.15.50:0/1445593963 shutdown_connections 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.715+0000 7fb38e427740 1 -- 172.21.15.50:0/1445593963 wait complete. 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 -- start start 2024-05-17T04:45:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 Processor -- start 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a137870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400e170 con 0x558f79ce52f0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400f5e0 con 0x558f79ced4d0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.716+0000 7fb38e427740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb36400e840 con 0x558f79d0e1b0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb38e427740 10 client.0 _ll_register_callbacks cb 0x7fb37c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb38522b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:33746/0 (socket says 172.21.15.50:33746) 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 -- 172.21.15.50:0/2953567504 learned_addr learned my addr 172.21.15.50:0/2953567504 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb385a2c640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 -- 172.21.15.50:0/2953567504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 msgr2=0x558f7a137870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a137870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 -- 172.21.15.50:0/2953567504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 msgr2=0x558f7a0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 -- 172.21.15.50:0/2953567504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f7a0caf40 con 0x558f79ced4d0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb38522b640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb384a2a640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb370005940 tx=0x7fb37003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4f7ced31ea206b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.717+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370042070 con 0x558f79ced4d0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb370004040 con 0x558f79ced4d0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37c0a52a0 con 0x558f79ced4d0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb370061050 con 0x558f79ced4d0 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 10 client.25787 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558f7a138e40 con 0x558f79ced4d0 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb37c0a52a0 con 0x558f79ced4d0 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb370051ce0 con 0x558f79ced4d0 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 10 client.25787 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 20 client.25787 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79ca21c0 0x558f7a13b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 4 client.25787 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f7a01f3f0 con 0x558f79ca21c0 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.718+0000 7fb38522b640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79ca21c0 0x558f7a13b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.719+0000 7fb368ff9640 10 client.25787 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.719+0000 7fb38522b640 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79ca21c0 0x558f7a13b880 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec034f5bea9924e server_cookie=2b9644f7febb87b2 in_seq=0 out_seq=0 2024-05-17T04:45:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.744+0000 7fb368ff9640 1 -- 172.21.15.50:0/2953567504 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f7a01f3f0 con 0x558f79ca21c0 2024-05-17T04:45:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.744+0000 7fb368ff9640 10 client.25787 handle_command_reply: tid=0 2024-05-17T04:45:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 client.25787 shutdown 2024-05-17T04:45:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 msgr2=0x558f7a0c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb370005940 tx=0x7fb37003f000 comp rx=0 tx=0).stop 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 shutdown_connections 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f79d0e1b0 0x558f7a137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f79ced4d0 0x558f7a0c7f50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x558f79ca21c0 0x558f7a13b880 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 --2- 172.21.15.50:0/2953567504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558f79ce52f0 0x558f7a0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 >> 172.21.15.50:0/2953567504 conn(0x558f79d02300 msgr2=0x558f7a0ab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb368ff9640 0 client.25787 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 shutdown_connections 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 1 -- 172.21.15.50:0/2953567504 wait complete. 2024-05-17T04:45:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.745+0000 7fb38e427740 20 client.25787 trim_cache size 0 max 16384 2024-05-17T04:45:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:45:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:45:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 msgr2=0x7fb36c0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 0x7fb36c0b4c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a950 tx=0x7fb3700058b0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 msgr2=0x7fb38019ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb38019ba30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb37c005fd0 tx=0x7fb37c00ec20 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 shutdown_connections 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb36c0b2770 0x7fb36c0b4c80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb380104730 0x7fb3801a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb380101ac0 0x7fb38019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 --2- 172.21.15.50:0/3984180029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb380101170 0x7fb38019ba30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 >> 172.21.15.50:0/3984180029 conn(0x7fb3800fd010 msgr2=0x7fb380102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 shutdown_connections 2024-05-17T04:45:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:34.746+0000 7fb38748f640 1 -- 172.21.15.50:0/3984180029 wait complete. 2024-05-17T04:45:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:36 smithi105 ceph-mon[34795]: pgmap v717: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 296 op/s 2024-05-17T04:45:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:36 smithi110 ceph-mon[35497]: pgmap v717: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 296 op/s 2024-05-17T04:45:36.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:36 smithi050 ceph-mon[31739]: pgmap v717: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.5 MiB/s wr, 296 op/s 2024-05-17T04:45:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 400.0 s, 105.0 tps, lat 340.741 ms stddev 82.379 2024-05-17T04:45:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:38 smithi105 ceph-mon[34795]: pgmap v718: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-17T04:45:38.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:38 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:45:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:38 smithi110 ceph-mon[35497]: pgmap v718: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-17T04:45:38.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:38 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:45:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:38 smithi050 ceph-mon[31739]: pgmap v718: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.4 MiB/s wr, 302 op/s 2024-05-17T04:45:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:38 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:45:39.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:39 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:39 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:39.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:39 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:40 smithi105 ceph-mon[34795]: pgmap v719: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 276 op/s 2024-05-17T04:45:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:40 smithi110 ceph-mon[35497]: pgmap v719: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 276 op/s 2024-05-17T04:45:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:40 smithi050 ceph-mon[31739]: pgmap v719: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.4 MiB/s wr, 276 op/s 2024-05-17T04:45:41.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: pgmap v720: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:45:41.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:45:41.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:45:41.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:41 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: pgmap v720: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:45:41.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:41 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:41.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: pgmap v720: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:45:41.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:45:41.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:45:41.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:41 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:45:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 405.0 s, 83.2 tps, lat 308.559 ms stddev 109.457 2024-05-17T04:45:44.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:44 smithi105 ceph-mon[34795]: pgmap v721: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-05-17T04:45:44.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:44 smithi110 ceph-mon[35497]: pgmap v721: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-05-17T04:45:44.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:44 smithi050 ceph-mon[31739]: pgmap v721: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-05-17T04:45:46.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:46 smithi105 ceph-mon[34795]: pgmap v722: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-17T04:45:46.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:46 smithi110 ceph-mon[35497]: pgmap v722: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-17T04:45:46.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:46 smithi050 ceph-mon[31739]: pgmap v722: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.6 MiB/s wr, 313 op/s 2024-05-17T04:45:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 410.0 s, 102.0 tps, lat 458.842 ms stddev 519.675 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 -- 172.21.15.50:0/3016741244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 msgr2=0x7f5b30115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/3016741244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b30115d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009860 tx=0x7f5b2802a3d0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 -- 172.21.15.50:0/3016741244 shutdown_connections 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/3016741244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b30115d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/3016741244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b3010cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/3016741244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b3010c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 -- 172.21.15.50:0/3016741244 >> 172.21.15.50:0/3016741244 conn(0x7f5b300701c0 msgr2=0x7f5b30072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 -- 172.21.15.50:0/3016741244 shutdown_connections 2024-05-17T04:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.919+0000 7f5b35ae7640 1 -- 172.21.15.50:0/3016741244 wait complete. 2024-05-17T04:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.920+0000 7f5b35ae7640 1 Processor -- start 2024-05-17T04:45:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.920+0000 7f5b35ae7640 1 -- start start 2024-05-17T04:45:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b301a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b301a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b301ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30119550 con 0x7f5b3010c880 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301193d0 con 0x7f5b3010d410 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b35ae7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301196d0 con 0x7f5b30109f20 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b301a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b301a41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44108/0 (socket says 172.21.15.50:44108) 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/2913334933 learned_addr learned my addr 172.21.15.50:0/2913334933 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b301ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.921+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b301a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 -- 172.21.15.50:0/2913334933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 msgr2=0x7f5b301a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b301a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 -- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 msgr2=0x7f5b301a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b301a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 -- 172.21.15.50:0/2913334933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b301abce0 con 0x7f5b3010d410 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b301a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b301ab670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802a8f0 tx=0x7f5b28005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2cff9640 1 -- 172.21.15.50:0/2913334933 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28040070 con 0x7f5b3010d410 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2cff9640 1 -- 172.21.15.50:0/2913334933 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b28030d20 con 0x7f5b3010d410 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b2cff9640 1 -- 172.21.15.50:0/2913334933 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004c90 con 0x7f5b3010d410 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b301abf70 con 0x7f5b3010d410 2024-05-17T04:45:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.922+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301ac320 con 0x7f5b3010d410 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.925+0000 7f5b2cff9640 1 -- 172.21.15.50:0/2913334933 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5b28039040 con 0x7f5b3010d410 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 Processor -- start 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 -- start start 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fc2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bf54d0 0x563ed6fc9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ed6f2bcc0 con 0x563ed6bf54d0 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ed6f1b860 con 0x563ed6bed2f0 2024-05-17T04:45:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ed6f1f8a0 con 0x563ed6c161b0 2024-05-17T04:45:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44116/0 (socket says 172.21.15.50:44116) 2024-05-17T04:45:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fc2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b2effd640 1 -- 172.21.15.50:0/2113278071 learned_addr learned my addr 172.21.15.50:0/2113278071 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.926+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bf54d0 0x563ed6fc9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 -- 172.21.15.50:0/2113278071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 msgr2=0x563ed6fc2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fc2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 -- 172.21.15.50:0/2113278071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bf54d0 msgr2=0x563ed6fc9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bf54d0 0x563ed6fc9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 -- 172.21.15.50:0/2113278071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed6ee3d50 con 0x563ed6c161b0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2cff9640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 0x7f5b1c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2effd640 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b18012850 tx=0x7f5b18009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b916c5fc2aa17e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 0x7f5b1c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b2cff9640 1 -- 172.21.15.50:0/2913334933 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b28103670 con 0x7f5b3010d410 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b25ffb640 1 -- 172.21.15.50:0/2113278071 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18012d10 con 0x563ed6c161b0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b25ffb640 1 -- 172.21.15.50:0/2113278071 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b18010c60 con 0x563ed6c161b0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.927+0000 7f5b25ffb640 1 -- 172.21.15.50:0/2113278071 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18012d10 con 0x563ed6c161b0 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2113278071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 msgr2=0x563ed6fc1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b18012850 tx=0x7f5b18009760 comp rx=0 tx=0).stop 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 0x7f5b1c0b4cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563ed6fc3380 tx=0x7f5b20009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2113278071 shutdown_connections 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bf54d0 0x563ed6fc9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fc2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2113278071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6c161b0 0x563ed6fc1f80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.928+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2113278071 >> 172.21.15.50:0/2113278071 conn(0x563ed6c0a300 msgr2=0x563ed6fbd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.929+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2113278071 shutdown_connections 2024-05-17T04:45:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.929+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2113278071 wait complete. 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 -- start start 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 Processor -- start 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fcb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fcbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 0x563ed6fd2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ed6e57ab0 con 0x563ed6bed2f0 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.930+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ed6ed84d0 con 0x563ed6c161b0 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563ed6fc9e40 con 0x563ed6bf54d0 2024-05-17T04:45:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fcb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fcb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39206/0 (socket says 172.21.15.50:39206) 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/985345693 learned_addr learned my addr 172.21.15.50:0/985345693 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 0x563ed6fd2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b3c99b740 10 client.0 _ll_register_callbacks cb 0x563ed6f16950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fcbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 -- 172.21.15.50:0/985345693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 msgr2=0x563ed6fcbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fcbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 -- 172.21.15.50:0/985345693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 msgr2=0x563ed6fcb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fcb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 -- 172.21.15.50:0/985345693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed6ed9340 con 0x563ed6c161b0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 0x563ed6fd2d20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005cd0 tx=0x7f5b280ca310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4cd4805af355eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.931+0000 7f5b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28032e60 con 0x563ed6c161b0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b18010c60 con 0x563ed6c161b0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b280ed0b0 con 0x563ed6c161b0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5b280ca410 con 0x563ed6c161b0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b3c99b740 10 client.25799 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563ed6c81790 con 0x563ed6c161b0 2024-05-17T04:45:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.932+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b18010c60 con 0x563ed6c161b0 2024-05-17T04:45:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5b2804d030 con 0x563ed6c161b0 2024-05-17T04:45:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 10 client.25799 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 10 client.25799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 20 client.25799 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6bd4d80 0x563ed6fd9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 4 client.25799 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6bd4d80 0x563ed6fd9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.933+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed6f2dc10 con 0x563ed6bd4d80 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.934+0000 7f5b0effd640 10 client.25799 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.934+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6bd4d80 0x563ed6fd9090 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7041880cfb91b626 server_cookie=77a6027b09436b87 in_seq=0 out_seq=0 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.935+0000 7f5b0effd640 1 -- 172.21.15.50:0/985345693 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ed6f2dc10 con 0x563ed6bd4d80 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.935+0000 7f5b0effd640 10 client.25799 handle_command_reply: tid=0 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.935+0000 7f5b3c99b740 1 client.25799 shutdown 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.935+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 msgr2=0x563ed6fd2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.935+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 0x563ed6fd2d20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005cd0 tx=0x7f5b280ca310 comp rx=0 tx=0).stop 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 shutdown_connections 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6c161b0 0x563ed6fd2d20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fcbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6bd4d80 0x563ed6fd9090 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/985345693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fcb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b0effd640 0 client.25799 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 >> 172.21.15.50:0/985345693 conn(0x563ed6c0a300 msgr2=0x563ed6fb41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 shutdown_connections 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 1 -- 172.21.15.50:0/985345693 wait complete. 2024-05-17T04:45:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.936+0000 7f5b3c99b740 20 client.25799 trim_cache size 0 max 16384 2024-05-17T04:45:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 Processor -- start 2024-05-17T04:45:47.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 -- start start 2024-05-17T04:45:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fd2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fb3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b4cc0 con 0x563ed6bed2f0 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b2590 con 0x563ed6bd4d80 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.958+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b03e0 con 0x563ed6bf54d0 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fd2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:41080/0 (socket says 172.21.15.50:41080) 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fd2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39218/0 (socket says 172.21.15.50:39218) 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/2687224001 learned_addr learned my addr 172.21.15.50:0/2687224001 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fb3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/2687224001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 msgr2=0x563ed6fb3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fb3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/2687224001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 msgr2=0x563ed6fd2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fd2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/2687224001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed6cfa620 con 0x563ed6bd4d80 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.959+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000f9b0 tx=0x7f5b2001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5dff0478ab2c281 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.960+0000 7f5b257fa640 1 -- 172.21.15.50:0/2687224001 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000fd60 con 0x563ed6bd4d80 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.960+0000 7f5b257fa640 1 -- 172.21.15.50:0/2687224001 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b20030070 con 0x563ed6bd4d80 2024-05-17T04:45:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.960+0000 7f5b257fa640 1 -- 172.21.15.50:0/2687224001 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004590 con 0x563ed6bd4d80 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2687224001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 msgr2=0x563ed6fbd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000f9b0 tx=0x7f5b2001b000 comp rx=0 tx=0).stop 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2687224001 shutdown_connections 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fb3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bed2f0 0x563ed6fd2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/2687224001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bd4d80 0x563ed6fbd2e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2687224001 >> 172.21.15.50:0/2687224001 conn(0x563ed6c0a300 msgr2=0x563ed6fb37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2687224001 shutdown_connections 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- 172.21.15.50:0/2687224001 wait complete. 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.961+0000 7f5b3c99b740 1 -- start start 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 Processor -- start 2024-05-17T04:45:47.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 0x563ed6fcfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fd03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fd12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b0a50 con 0x563ed6bd4d80 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b1ec0 con 0x563ed6bed2f0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b301b1120 con 0x563ed6bf54d0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 0x563ed6fcfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fd03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b2f7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 0x563ed6fcfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39220/0 (socket says 172.21.15.50:39220) 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b2f7fe640 1 -- 172.21.15.50:0/3906305666 learned_addr learned my addr 172.21.15.50:0/3906305666 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b3c99b740 10 client.0 _ll_register_callbacks cb 0x7f5b28005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.962+0000 7f5b2ffff640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fd12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 -- 172.21.15.50:0/3906305666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 msgr2=0x563ed6fd12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fd12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 -- 172.21.15.50:0/3906305666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 msgr2=0x563ed6fcfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 0x563ed6fcfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 -- 172.21.15.50:0/3906305666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed6fd18a0 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b2effd640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fd03f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006010 tx=0x7f5b180109d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f2330b7e21faf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1802c060 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b20030070 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5b18024040 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.963+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5b180055f0 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 10 client.25805 fetch_fsmap learned FSMap version 39 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563ed6fd2b80 con 0x563ed6bed2f0 2024-05-17T04:45:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5b20030070 con 0x563ed6bed2f0 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5b180049f0 con 0x563ed6bed2f0 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 10 client.25805 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 20 client.25805 populate_metadata read hostname 'smithi050' 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6c161b0 0x563ed7043560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 4 client.25805 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ed6f273f0 con 0x563ed6c161b0 2024-05-17T04:45:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.964+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6c161b0 0x563ed7043560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:45:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.966+0000 7f5b0effd640 10 client.25805 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.966+0000 7f5b2f7fe640 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6c161b0 0x563ed7043560 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c7d76c05045670 server_cookie=7dd924a07e84813a in_seq=0 out_seq=0 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b0effd640 1 -- 172.21.15.50:0/3906305666 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ed6f273f0 con 0x563ed6c161b0 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b0effd640 10 client.25805 handle_command_reply: tid=0 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 client.25805 shutdown 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 msgr2=0x563ed6fd03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fd03f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006010 tx=0x7f5b180109d0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 shutdown_connections 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563ed6bf54d0 0x563ed6fd12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ed6bed2f0 0x563ed6fd03f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x563ed6c161b0 0x563ed7043560 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 --2- 172.21.15.50:0/3906305666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed6bd4d80 0x563ed6fcfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b0effd640 0 client.25805 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 >> 172.21.15.50:0/3906305666 conn(0x563ed6c0a300 msgr2=0x563ed6fb41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.989+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 shutdown_connections 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b3c99b740 1 -- 172.21.15.50:0/3906305666 wait complete. 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b3c99b740 20 client.25805 trim_cache size 0 max 16384 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:45:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:45:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 msgr2=0x7f5b1c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 0x7f5b1c0b4cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563ed6fc3380 tx=0x7f5b20009500 comp rx=0 tx=0).stop 2024-05-17T04:45:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 msgr2=0x7f5b301ab670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:45:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b301ab670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b2802a8f0 tx=0x7f5b28005270 comp rx=0 tx=0).stop 2024-05-17T04:45:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 shutdown_connections 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b3010d410 0x7f5b301ab670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5b1c0b27e0 0x7f5b1c0b4cf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5b3010c880 0x7f5b301a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 --2- 172.21.15.50:0/2913334933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b30109f20 0x7f5b301a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.990+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 >> 172.21.15.50:0/2913334933 conn(0x7f5b300701c0 msgr2=0x7f5b30111d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.991+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 shutdown_connections 2024-05-17T04:45:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:45:47.991+0000 7f5b35ae7640 1 -- 172.21.15.50:0/2913334933 wait complete. 2024-05-17T04:45:48.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:48 smithi105 ceph-mon[34795]: pgmap v723: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-05-17T04:45:48.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:48 smithi110 ceph-mon[35497]: pgmap v723: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-05-17T04:45:48.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:48 smithi050 ceph-mon[31739]: pgmap v723: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.3 MiB/s wr, 313 op/s 2024-05-17T04:45:50.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:50 smithi105 ceph-mon[34795]: pgmap v724: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-05-17T04:45:50.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:50 smithi110 ceph-mon[35497]: pgmap v724: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-05-17T04:45:50.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:50 smithi050 ceph-mon[31739]: pgmap v724: 129 pgs: 129 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.8 MiB/s wr, 255 op/s 2024-05-17T04:45:52.315 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:52 smithi105 ceph-mon[34795]: pgmap v725: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-17T04:45:52.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:52 smithi110 ceph-mon[35497]: pgmap v725: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-17T04:45:52.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:52 smithi050 ceph-mon[31739]: pgmap v725: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-17T04:45:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 415.0 s, 113.6 tps, lat 321.108 ms stddev 93.960 2024-05-17T04:45:54.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:54 smithi105 ceph-mon[34795]: pgmap v726: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-05-17T04:45:54.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:54 smithi110 ceph-mon[35497]: pgmap v726: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-05-17T04:45:54.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:54 smithi050 ceph-mon[31739]: pgmap v726: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.4 MiB/s wr, 332 op/s 2024-05-17T04:45:56.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:56 smithi105 ceph-mon[34795]: pgmap v727: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-17T04:45:56.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:56 smithi110 ceph-mon[35497]: pgmap v727: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-17T04:45:56.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:56 smithi050 ceph-mon[31739]: pgmap v727: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-05-17T04:45:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 420.0 s, 129.6 tps, lat 265.584 ms stddev 91.224 2024-05-17T04:45:58.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:45:58 smithi105 ceph-mon[34795]: pgmap v728: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-05-17T04:45:58.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:45:58 smithi110 ceph-mon[35497]: pgmap v728: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-05-17T04:45:58.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:45:58 smithi050 ceph-mon[31739]: pgmap v728: 129 pgs: 129 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-05-17T04:46:00.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:00 smithi105 ceph-mon[34795]: pgmap v729: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-05-17T04:46:00.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:00 smithi110 ceph-mon[35497]: pgmap v729: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-05-17T04:46:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:00 smithi050 ceph-mon[31739]: pgmap v729: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-05-17T04:46:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.159+0000 7fd6888df640 1 -- 172.21.15.50:0/2471942536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410db80 msgr2=0x7fd68410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 --2- 172.21.15.50:0/2471942536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410db80 0x7fd68410df90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd674009ca0 tx=0x7fd67402a8c0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 -- 172.21.15.50:0/2471942536 shutdown_connections 2024-05-17T04:46:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 --2- 172.21.15.50:0/2471942536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410f5f0 0x7fd684115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 --2- 172.21.15.50:0/2471942536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410eca0 0x7fd68410f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 --2- 172.21.15.50:0/2471942536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410db80 0x7fd68410df90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 -- 172.21.15.50:0/2471942536 >> 172.21.15.50:0/2471942536 conn(0x7fd68406dd20 msgr2=0x7fd68410bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 -- 172.21.15.50:0/2471942536 shutdown_connections 2024-05-17T04:46:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.160+0000 7fd6888df640 1 -- 172.21.15.50:0/2471942536 wait complete. 2024-05-17T04:46:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.161+0000 7fd6888df640 1 Processor -- start 2024-05-17T04:46:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.161+0000 7fd6888df640 1 -- start start 2024-05-17T04:46:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 0x7fd6841a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd684119570 con 0x7fd68410eca0 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841193f0 con 0x7fd68410f5f0 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd6888df640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841196f0 con 0x7fd68410db80 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd68259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd682d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd682d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56406/0 (socket says 172.21.15.50:56406) 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd682d9d640 1 -- 172.21.15.50:0/3877999640 learned_addr learned my addr 172.21.15.50:0/3877999640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd68259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48728/0 (socket says 172.21.15.50:48728) 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.162+0000 7fd679d9b640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 0x7fd6841a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 -- 172.21.15.50:0/3877999640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 msgr2=0x7fd6841ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 -- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 msgr2=0x7fd6841a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 0x7fd6841a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 -- 172.21.15.50:0/3877999640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6841abd80 con 0x7fd68410db80 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd682d9d640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd679d9b640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 0x7fd6841a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd68259c640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd67402aeb0 tx=0x7fd6740093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd67bfff640 1 -- 172.21.15.50:0/3877999640 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674003cf0 con 0x7fd68410db80 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.163+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841ac010 con 0x7fd68410db80 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.164+0000 7fd67bfff640 1 -- 172.21.15.50:0/3877999640 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6740044d0 con 0x7fd68410db80 2024-05-17T04:46:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.164+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841ac5b0 con 0x7fd68410db80 2024-05-17T04:46:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.164+0000 7fd67bfff640 1 -- 172.21.15.50:0/3877999640 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674030b70 con 0x7fd68410db80 2024-05-17T04:46:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.167+0000 7fd67bfff640 1 -- 172.21.15.50:0/3877999640 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd67404e440 con 0x7fd68410db80 2024-05-17T04:46:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.168+0000 7fd68f799740 1 Processor -- start 2024-05-17T04:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.168+0000 7fd68f799740 1 -- start start 2024-05-17T04:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.168+0000 7fd68f799740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304611b0 0x56073080cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68f799740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68f799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 0x560730814420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68f799740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560730776cc0 con 0x5607304382f0 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68f799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560730766860 con 0x5607304404d0 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68f799740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56073076a8a0 con 0x5607304611b0 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54436/0 (socket says 172.21.15.50:54436) 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd68259c640 1 -- 172.21.15.50:0/655858135 learned_addr learned my addr 172.21.15.50:0/655858135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd679d9b640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 0x560730814420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd682d9d640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304611b0 0x56073080cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.169+0000 7fd67bfff640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 0x7fd6700b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 -- 172.21.15.50:0/655858135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304611b0 msgr2=0x56073080cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304611b0 0x56073080cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 -- 172.21.15.50:0/655858135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 msgr2=0x56073080d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 -- 172.21.15.50:0/655858135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073072ed50 con 0x5607304404d0 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd68259c640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd67bfff640 1 -- 172.21.15.50:0/3877999640 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd6741039b0 con 0x7fd68410db80 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd679d9b640 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 0x560730814420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005d00 tx=0x7fd66c00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f03499fa5c5358 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd682d9d640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 0x7fd6700b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.170+0000 7fd678d99640 1 -- 172.21.15.50:0/655858135 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c016070 con 0x5607304404d0 2024-05-17T04:46:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.171+0000 7fd678d99640 1 -- 172.21.15.50:0/655858135 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd66c0035f0 con 0x5607304404d0 2024-05-17T04:46:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.171+0000 7fd678d99640 1 -- 172.21.15.50:0/655858135 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0116b0 con 0x5607304404d0 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 -- 172.21.15.50:0/655858135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 msgr2=0x560730814420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 0x560730814420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005d00 tx=0x7fd66c00e6c0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd682d9d640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 0x7fd6700b4d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0095f0 tx=0x7fd67c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 -- 172.21.15.50:0/655858135 shutdown_connections 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304404d0 0x560730814420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073080d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 --2- 172.21.15.50:0/655858135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304611b0 0x56073080cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.172+0000 7fd68f799740 1 -- 172.21.15.50:0/655858135 >> 172.21.15.50:0/655858135 conn(0x560730455300 msgr2=0x5607308089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.173+0000 7fd68f799740 1 -- 172.21.15.50:0/655858135 shutdown_connections 2024-05-17T04:46:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.173+0000 7fd68f799740 1 -- 172.21.15.50:0/655858135 wait complete. 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.173+0000 7fd68f799740 1 -- start start 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.173+0000 7fd68f799740 1 Processor -- start 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x5607308168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x560730816e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5607306a2ab0 con 0x5607304611b0 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5607307234d0 con 0x5607304382f0 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560730814f40 con 0x5607304404d0 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd682d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68f799740 10 client.0 _ll_register_callbacks cb 0x560730761950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x5607308168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x5607308168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56424/0 (socket says 172.21.15.50:56424) 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd68259c640 1 -- 172.21.15.50:0/213137870 learned_addr learned my addr 172.21.15.50:0/213137870 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd682d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54438/0 (socket says 172.21.15.50:54438) 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.174+0000 7fd679d9b640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x560730816e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 -- 172.21.15.50:0/213137870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 msgr2=0x560730816e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x560730816e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 -- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 msgr2=0x5607308168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x5607308168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 -- 172.21.15.50:0/213137870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560730724340 con 0x5607304611b0 2024-05-17T04:46:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd682d9d640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x56073080e170 tx=0x7fd67c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4035a812073663 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.175+0000 7fd669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c010640 con 0x5607304611b0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd67c018050 con 0x5607304611b0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66c0035f0 con 0x5607304611b0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd67c0107e0 con 0x5607304611b0 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd68f799740 10 client.16059 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5607304cc790 con 0x5607304611b0 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.176+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd67c010e10 con 0x5607304611b0 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd67c01a070 con 0x5607304611b0 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 10 client.16059 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 20 client.16059 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56073041fd80 0x560730824580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 4 client.16059 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:46:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd66c0035f0 con 0x56073041fd80 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.177+0000 7fd68259c640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56073041fd80 0x560730824580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.178+0000 7fd669ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.178+0000 7fd68259c640 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56073041fd80 0x560730824580 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb7b29a8d7fb363 server_cookie=14a04093f760893 in_seq=0 out_seq=0 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.178+0000 7fd669ffb640 1 -- 172.21.15.50:0/213137870 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd66c0035f0 con 0x56073041fd80 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.179+0000 7fd669ffb640 10 client.16059 handle_command_reply: tid=0 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.179+0000 7fd68f799740 1 client.16059 shutdown 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 msgr2=0x56073081dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x56073080e170 tx=0x7fd67c005f90 comp rx=0 tx=0).stop 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 shutdown_connections 2024-05-17T04:46:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56073041fd80 0x560730824580 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x56073081dd60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x560730816e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 --2- 172.21.15.50:0/213137870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x5607308168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 >> 172.21.15.50:0/213137870 conn(0x560730455300 msgr2=0x56073080b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd669ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 shutdown_connections 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.180+0000 7fd68f799740 1 -- 172.21.15.50:0/213137870 wait complete. 2024-05-17T04:46:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.181+0000 7fd68f799740 20 client.16059 trim_cache size 0 max 16384 2024-05-17T04:46:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 Processor -- start 2024-05-17T04:46:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 -- start start 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x560730808240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x560730808780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b4fc0 con 0x5607304611b0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b2890 con 0x5607304382f0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.209+0000 7fd68f799740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b0700 con 0x5607304404d0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48756/0 (socket says 172.21.15.50:48756) 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 -- 172.21.15.50:0/2529108219 learned_addr learned my addr 172.21.15.50:0/2529108219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd68259c640 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x560730808240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd682d9d640 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x560730808780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 -- 172.21.15.50:0/2529108219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 msgr2=0x560730808240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x560730808240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 -- 172.21.15.50:0/2529108219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 msgr2=0x560730808780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x560730808780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 -- 172.21.15.50:0/2529108219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560730545620 con 0x5607304404d0 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.210+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005d00 tx=0x7fd66c0099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4b710abcaa70a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.211+0000 7fd66bfff640 1 -- 172.21.15.50:0/2529108219 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c016040 con 0x5607304404d0 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.211+0000 7fd66bfff640 1 -- 172.21.15.50:0/2529108219 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd66c029070 con 0x5607304404d0 2024-05-17T04:46:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.211+0000 7fd66bfff640 1 -- 172.21.15.50:0/2529108219 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c003b20 con 0x5607304404d0 2024-05-17T04:46:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.212+0000 7fd68f799740 1 -- 172.21.15.50:0/2529108219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 msgr2=0x56073081dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.212+0000 7fd68f799740 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005d00 tx=0x7fd66c0099c0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 -- 172.21.15.50:0/2529108219 shutdown_connections 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304611b0 0x560730808780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dcd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 --2- 172.21.15.50:0/2529108219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304382f0 0x560730808240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 -- 172.21.15.50:0/2529108219 >> 172.21.15.50:0/2529108219 conn(0x560730455300 msgr2=0x56073080cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 -- 172.21.15.50:0/2529108219 shutdown_connections 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 -- 172.21.15.50:0/2529108219 wait complete. 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.213+0000 7fd68f799740 1 -- start start 2024-05-17T04:46:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 Processor -- start 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304611b0 0x56073081a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b0d50 con 0x5607304382f0 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b21c0 con 0x5607304611b0 2024-05-17T04:46:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841b1420 con 0x5607304404d0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68f799740 10 client.0 _ll_register_callbacks cb 0x7fd67c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.214+0000 7fd68259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54444/0 (socket says 172.21.15.50:54444) 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 -- 172.21.15.50:0/2946225403 learned_addr learned my addr 172.21.15.50:0/2946225403 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 -- 172.21.15.50:0/2946225403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 msgr2=0x56073081dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 -- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304611b0 msgr2=0x56073081a400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304611b0 0x56073081a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 -- 172.21.15.50:0/2946225403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073081ab90 con 0x5607304382f0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.215+0000 7fd68259c640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd6740cabd0 tx=0x7fd6741094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393129fbb0b2d4cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.216+0000 7fd669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.216+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674103e60 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.216+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd674109120 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.216+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd66c029070 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.217+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd6740d33e0 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.217+0000 7fd68f799740 10 client.16065 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.217+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56073088b730 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd674036040 con 0x5607304382f0 2024-05-17T04:46:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd6740c6ed0 con 0x5607304382f0 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 10 client.16065 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 20 client.16065 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607303f51c0 0x56073088e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 4 client.16065 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd66c029070 con 0x5607303f51c0 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.218+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607303f51c0 0x56073088e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.219+0000 7fd669ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.219+0000 7fd679d9b640 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607303f51c0 0x56073088e080 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec013f64dbfa932 server_cookie=dcfa299fe3a05b68 in_seq=0 out_seq=0 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd669ffb640 1 -- 172.21.15.50:0/2946225403 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fd66c029070 con 0x5607303f51c0 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd669ffb640 10 client.16065 handle_command_reply: tid=0 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd68f799740 1 client.16065 shutdown 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 msgr2=0x56073081d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd6740cabd0 tx=0x7fd6741094f0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.243+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 shutdown_connections 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607304611b0 0x56073081a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607304404d0 0x56073081dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5607303f51c0 0x56073088e080 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 --2- 172.21.15.50:0/2946225403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607304382f0 0x56073081d7e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 >> 172.21.15.50:0/2946225403 conn(0x560730455300 msgr2=0x560730816580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd669ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 shutdown_connections 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 1 -- 172.21.15.50:0/2946225403 wait complete. 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd68f799740 20 client.16065 trim_cache size 0 max 16384 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:46:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:46:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 msgr2=0x7fd6700b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.244+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 0x7fd6700b4d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0095f0 tx=0x7fd67c0046c0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 msgr2=0x7fd6841a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd67402aeb0 tx=0x7fd6740093e0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 shutdown_connections 2024-05-17T04:46:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd68410f5f0 0x7fd6841ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd6700b2860 0x7fd6700b4d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd68410eca0 0x7fd6841a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 --2- 172.21.15.50:0/3877999640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd68410db80 0x7fd6841a4240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 >> 172.21.15.50:0/3877999640 conn(0x7fd68406dd20 msgr2=0x7fd684111fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 shutdown_connections 2024-05-17T04:46:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:01.245+0000 7fd6888df640 1 -- 172.21.15.50:0/3877999640 wait complete. 2024-05-17T04:46:02.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:02 smithi105 ceph-mon[34795]: pgmap v730: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-17T04:46:02.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:02 smithi110 ceph-mon[35497]: pgmap v730: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-17T04:46:02.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:02 smithi050 ceph-mon[31739]: pgmap v730: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.9 MiB/s wr, 332 op/s 2024-05-17T04:46:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 425.0 s, 142.4 tps, lat 10817.858 ms stddev 33246.732 2024-05-17T04:46:04.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:04 smithi105 ceph-mon[34795]: pgmap v731: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-17T04:46:04.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:04 smithi110 ceph-mon[35497]: pgmap v731: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-17T04:46:04.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:04 smithi050 ceph-mon[31739]: pgmap v731: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.5 MiB/s wr, 329 op/s 2024-05-17T04:46:06.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:06 smithi105 ceph-mon[34795]: pgmap v732: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-05-17T04:46:06.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:06 smithi110 ceph-mon[35497]: pgmap v732: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-05-17T04:46:06.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:06 smithi050 ceph-mon[31739]: pgmap v732: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-05-17T04:46:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 430.0 s, 148.4 tps, lat 689.925 ms stddev 201.865 2024-05-17T04:46:08.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:08 smithi105 ceph-mon[34795]: pgmap v733: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-05-17T04:46:08.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:08 smithi110 ceph-mon[35497]: pgmap v733: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-05-17T04:46:08.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:08 smithi050 ceph-mon[31739]: pgmap v733: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.0 MiB/s wr, 259 op/s 2024-05-17T04:46:10.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:10 smithi105 ceph-mon[34795]: pgmap v734: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 328 op/s 2024-05-17T04:46:10.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:10 smithi110 ceph-mon[35497]: pgmap v734: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 328 op/s 2024-05-17T04:46:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:10 smithi050 ceph-mon[31739]: pgmap v734: 129 pgs: 129 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.6 MiB/s wr, 328 op/s 2024-05-17T04:46:12.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:12 smithi105 ceph-mon[34795]: pgmap v735: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:46:12.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:12 smithi110 ceph-mon[35497]: pgmap v735: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:46:12.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:12 smithi050 ceph-mon[31739]: pgmap v735: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-05-17T04:46:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 435.0 s, 135.8 tps, lat 701.349 ms stddev 216.999 2024-05-17T04:46:14.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:14 smithi105 ceph-mon[34795]: pgmap v736: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-17T04:46:14.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:14 smithi110 ceph-mon[35497]: pgmap v736: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-17T04:46:14.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:14 smithi050 ceph-mon[31739]: pgmap v736: 129 pgs: 129 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-05-17T04:46:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 -- 172.21.15.50:0/210788836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 msgr2=0x7fc37c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 --2- 172.21.15.50:0/210788836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c101c20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc370009900 tx=0x7fc37002a410 comp rx=0 tx=0).stop 2024-05-17T04:46:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 -- 172.21.15.50:0/210788836 shutdown_connections 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 --2- 172.21.15.50:0/210788836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 0x7fc37c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 --2- 172.21.15.50:0/210788836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 0x7fc37c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 --2- 172.21.15.50:0/210788836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c101c20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.371+0000 7fc382bba640 1 -- 172.21.15.50:0/210788836 >> 172.21.15.50:0/210788836 conn(0x7fc37c0fd010 msgr2=0x7fc37c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.372+0000 7fc382bba640 1 -- 172.21.15.50:0/210788836 shutdown_connections 2024-05-17T04:46:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.372+0000 7fc382bba640 1 -- 172.21.15.50:0/210788836 wait complete. 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.372+0000 7fc382bba640 1 Processor -- start 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.372+0000 7fc382bba640 1 -- start start 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 0x7fc37c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 0x7fc37c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d1b0 con 0x7fc37c101810 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d030 con 0x7fc37c103280 2024-05-17T04:46:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc382bba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c10d330 con 0x7fc37c102930 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:48784/0 (socket says 172.21.15.50:48784) 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 -- 172.21.15.50:0/3259937798 learned_addr learned my addr 172.21.15.50:0/3259937798 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc37bfff640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 0x7fc37c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc381157640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 0x7fc37c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 -- 172.21.15.50:0/3259937798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 msgr2=0x7fc37c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 0x7fc37c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 -- 172.21.15.50:0/3259937798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 msgr2=0x7fc37c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 0x7fc37c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 -- 172.21.15.50:0/3259937798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1a1330 con 0x7fc37c101810 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.373+0000 7fc380956640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc370005ab0 tx=0x7fc370004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.374+0000 7fc379ffb640 1 -- 172.21.15.50:0/3259937798 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37003f030 con 0x7fc37c101810 2024-05-17T04:46:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.374+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1a15c0 con 0x7fc37c101810 2024-05-17T04:46:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.374+0000 7fc379ffb640 1 -- 172.21.15.50:0/3259937798 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc370004440 con 0x7fc37c101810 2024-05-17T04:46:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.374+0000 7fc379ffb640 1 -- 172.21.15.50:0/3259937798 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370035070 con 0x7fc37c101810 2024-05-17T04:46:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.374+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a1ab0 con 0x7fc37c101810 2024-05-17T04:46:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.377+0000 7fc389a71740 1 Processor -- start 2024-05-17T04:46:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.377+0000 7fc389a71740 1 -- start start 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.377+0000 7fc389a71740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.377+0000 7fc389a71740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc389a71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 0x56425ff1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc389a71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56425fe85cc0 con 0x56425fb4f4d0 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc389a71740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56425fe75860 con 0x56425fb701b0 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc389a71740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56425fe798a0 con 0x56425fb472f0 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39592/0 (socket says 172.21.15.50:39592) 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 -- 172.21.15.50:0/2261105166 learned_addr learned my addr 172.21.15.50:0/2261105166 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc379ffb640 1 -- 172.21.15.50:0/3259937798 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc370042030 con 0x7fc37c101810 2024-05-17T04:46:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc37bfff640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 0x56425ff1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 -- 172.21.15.50:0/2261105166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 msgr2=0x56425ff18100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc380956640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff18100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 -- 172.21.15.50:0/2261105166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 msgr2=0x56425ff1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 0x56425ff1f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 -- 172.21.15.50:0/2261105166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425fe3dd50 con 0x56425fb701b0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc37bfff640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 0x56425ff1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.378+0000 7fc381157640 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc374009fd0 tx=0x7fc3740118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e665212f593a630c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.379+0000 7fc366ffd640 1 -- 172.21.15.50:0/2261105166 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374004d20 con 0x56425fb701b0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.379+0000 7fc366ffd640 1 -- 172.21.15.50:0/2261105166 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc374004050 con 0x56425fb701b0 2024-05-17T04:46:14.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.379+0000 7fc366ffd640 1 -- 172.21.15.50:0/2261105166 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374005260 con 0x56425fb701b0 2024-05-17T04:46:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.379+0000 7fc379ffb640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 0x7fc36c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.379+0000 7fc379ffb640 1 -- 172.21.15.50:0/3259937798 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc370102990 con 0x7fc37c101810 2024-05-17T04:46:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc380956640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 0x7fc36c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 -- 172.21.15.50:0/2261105166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 msgr2=0x56425ff17b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.384 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc374009fd0 tx=0x7fc3740118d0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 -- 172.21.15.50:0/2261105166 shutdown_connections 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb4f4d0 0x56425ff1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff18100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 --2- 172.21.15.50:0/2261105166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb701b0 0x56425ff17b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 -- 172.21.15.50:0/2261105166 >> 172.21.15.50:0/2261105166 conn(0x56425fb64300 msgr2=0x56425ff0f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 -- 172.21.15.50:0/2261105166 shutdown_connections 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc389a71740 1 -- 172.21.15.50:0/2261105166 wait complete. 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.380+0000 7fc380956640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 0x7fc36c0b4d70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3700c5cf0 tx=0x7fc3700045f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 -- start start 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 Processor -- start 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56425fdb1ab0 con 0x56425fb701b0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56425fe324d0 con 0x56425fb4f4d0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.381+0000 7fc389a71740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56425ff1faa0 con 0x56425fb472f0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc389a71740 10 client.0 _ll_register_callbacks cb 0x56425fe70950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc381157640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc381157640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff21470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:53882/0 (socket says 172.21.15.50:53882) 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc381157640 1 -- 172.21.15.50:0/1967590816 learned_addr learned my addr 172.21.15.50:0/1967590816 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc381157640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 -- 172.21.15.50:0/1967590816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 msgr2=0x56425ff21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 -- 172.21.15.50:0/1967590816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 msgr2=0x56425ff28910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 -- 172.21.15.50:0/1967590816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425fe33340 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc37bfff640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff219b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0fea40 tx=0x7fc368012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612afe581696395a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.382+0000 7fc35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368004050 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc3680041f0 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc374004050 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc368025050 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 10 client.25832 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56425fbdb790 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc374004050 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc368003960 con 0x56425fb4f4d0 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 10 client.25832 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 20 client.25832 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.383+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb2ed80 0x56425ff2ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.384+0000 7fc389a71740 4 client.25832 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.384+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425fe87c10 con 0x56425fb2ed80 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.384+0000 7fc381157640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb2ed80 0x56425ff2ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.384+0000 7fc35bfff640 10 client.25832 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.384+0000 7fc381157640 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb2ed80 0x56425ff2ec90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972145e936671890 server_cookie=eff3e021283374ce in_seq=0 out_seq=0 2024-05-17T04:46:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc35bfff640 1 -- 172.21.15.50:0/1967590816 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56425fe87c10 con 0x56425fb2ed80 2024-05-17T04:46:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc35bfff640 10 client.25832 handle_command_reply: tid=0 2024-05-17T04:46:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 client.25832 shutdown 2024-05-17T04:46:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 msgr2=0x56425ff219b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff219b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0fea40 tx=0x7fc368012b60 comp rx=0 tx=0).stop 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 shutdown_connections 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb2ed80 0x56425ff2ec90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff219b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc35bfff640 0 client.25832 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 --2- 172.21.15.50:0/1967590816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.385+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 >> 172.21.15.50:0/1967590816 conn(0x56425fb64300 msgr2=0x56425ff12010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.386+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 shutdown_connections 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.386+0000 7fc389a71740 1 -- 172.21.15.50:0/1967590816 wait complete. 2024-05-17T04:46:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.386+0000 7fc389a71740 20 client.25832 trim_cache size 0 max 16384 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.409+0000 7fc389a71740 1 Processor -- start 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.409+0000 7fc389a71740 1 -- start start 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.409+0000 7fc389a71740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc389a71740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb4f4d0 0x56425ff28830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc389a71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc389a71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1ac6d0 con 0x56425fb701b0 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc389a71740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1a9fa0 con 0x56425fb472f0 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc389a71740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1a7df0 con 0x56425fb4f4d0 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39620/0 (socket says 172.21.15.50:39620) 2024-05-17T04:46:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 -- 172.21.15.50:0/3301619013 learned_addr learned my addr 172.21.15.50:0/3301619013 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc37bfff640 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb4f4d0 0x56425ff28830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 -- 172.21.15.50:0/3301619013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb4f4d0 msgr2=0x56425ff28830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb4f4d0 0x56425ff28830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 -- 172.21.15.50:0/3301619013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 msgr2=0x56425ff28d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 -- 172.21.15.50:0/3301619013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425fc54620 con 0x56425fb472f0 2024-05-17T04:46:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.410+0000 7fc381157640 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc374009f70 tx=0x7fc374009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2c4bf4c3af7625 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.411+0000 7fc3667fc640 1 -- 172.21.15.50:0/3301619013 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374003c30 con 0x56425fb472f0 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.411+0000 7fc3667fc640 1 -- 172.21.15.50:0/3301619013 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc37402e1c0 con 0x56425fb472f0 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.411+0000 7fc3667fc640 1 -- 172.21.15.50:0/3301619013 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37400f740 con 0x56425fb472f0 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- 172.21.15.50:0/3301619013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 msgr2=0x56425ff1bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc374009f70 tx=0x7fc374009500 comp rx=0 tx=0).stop 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- 172.21.15.50:0/3301619013 shutdown_connections 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff28d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb4f4d0 0x56425ff28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 --2- 172.21.15.50:0/3301619013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb472f0 0x56425ff1bfd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- 172.21.15.50:0/3301619013 >> 172.21.15.50:0/3301619013 conn(0x56425fb64300 msgr2=0x56425ff12510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- 172.21.15.50:0/3301619013 shutdown_connections 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- 172.21.15.50:0/3301619013 wait complete. 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 -- start start 2024-05-17T04:46:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.412+0000 7fc389a71740 1 Processor -- start 2024-05-17T04:46:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff27c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff94ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1a8460 con 0x56425fb701b0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1a98d0 con 0x56425fb4f4d0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1a8b30 con 0x56425fb472f0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc381157640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff27c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc389a71740 10 client.0 _ll_register_callbacks cb 0x7fc368009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc381157640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff27c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:53902/0 (socket says 172.21.15.50:53902) 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc381157640 1 -- 172.21.15.50:0/3756019270 learned_addr learned my addr 172.21.15.50:0/3756019270 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39628/0 (socket says 172.21.15.50:39628) 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc381157640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff94ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 -- 172.21.15.50:0/3756019270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 msgr2=0x56425ff27c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff27c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 -- 172.21.15.50:0/3756019270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 msgr2=0x56425ff94ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.413+0000 7fc37bfff640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff94ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc37bfff640 1 -- 172.21.15.50:0/3756019270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425ff28970 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc37bfff640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc368003ef0 tx=0x7fc368003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abdc2d8ed8f815d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc35bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368009bf0 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc368022070 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37402e1c0 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc368039050 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc389a71740 10 client.25838 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.414+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56425ff96590 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc36800fdb0 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc368010430 con 0x56425fb4f4d0 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 10 client.25838 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 20 client.25838 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb041c0 0x56425ff98fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 4 client.25838 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc37402e1c0 con 0x56425fb041c0 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.415+0000 7fc381157640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb041c0 0x56425ff98fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.416+0000 7fc35bfff640 10 client.25838 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.416+0000 7fc381157640 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb041c0 0x56425ff98fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26ca80aed699c83 server_cookie=6127996ed8f79157 in_seq=0 out_seq=0 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.448+0000 7fc35bfff640 1 -- 172.21.15.50:0/3756019270 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fc37402e1c0 con 0x56425fb041c0 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.448+0000 7fc35bfff640 10 client.25838 handle_command_reply: tid=0 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.448+0000 7fc389a71740 1 client.25838 shutdown 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.448+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 msgr2=0x56425ff28140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc368003ef0 tx=0x7fc368003820 comp rx=0 tx=0).stop 2024-05-17T04:46:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 shutdown_connections 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56425fb041c0 0x56425ff98fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56425fb701b0 0x56425ff94ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56425fb4f4d0 0x56425ff28140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 --2- 172.21.15.50:0/3756019270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56425fb472f0 0x56425ff27c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc35bfff640 0 client.25838 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 >> 172.21.15.50:0/3756019270 conn(0x56425fb64300 msgr2=0x56425ff122e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 shutdown_connections 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 1 -- 172.21.15.50:0/3756019270 wait complete. 2024-05-17T04:46:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.449+0000 7fc389a71740 20 client.25838 trim_cache size 0 max 16384 2024-05-17T04:46:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:46:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:46:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 msgr2=0x7fc36c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 0x7fc36c0b4d70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3700c5cf0 tx=0x7fc3700045f0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 msgr2=0x7fc37c0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc370005ab0 tx=0x7fc370004040 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 shutdown_connections 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc37c103280 0x7fc37c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc37c102930 0x7fc37c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc36c0b2860 0x7fc36c0b4d70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 --2- 172.21.15.50:0/3259937798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc37c101810 0x7fc37c0ff0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 >> 172.21.15.50:0/3259937798 conn(0x7fc37c0fd010 msgr2=0x7fc37c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 shutdown_connections 2024-05-17T04:46:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:14.450+0000 7fc382bba640 1 -- 172.21.15.50:0/3259937798 wait complete. 2024-05-17T04:46:16.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:16 smithi105 ceph-mon[34795]: pgmap v737: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.6 MiB/s wr, 295 op/s 2024-05-17T04:46:16.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:16 smithi110 ceph-mon[35497]: pgmap v737: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.6 MiB/s wr, 295 op/s 2024-05-17T04:46:16.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:16 smithi050 ceph-mon[31739]: pgmap v737: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.6 MiB/s wr, 295 op/s 2024-05-17T04:46:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 440.0 s, 134.8 tps, lat 759.500 ms stddev 223.346 2024-05-17T04:46:18.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:18 smithi105 ceph-mon[34795]: pgmap v738: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-17T04:46:18.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:18 smithi110 ceph-mon[35497]: pgmap v738: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-17T04:46:18.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:18 smithi050 ceph-mon[31739]: pgmap v738: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-17T04:46:20.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:20 smithi105 ceph-mon[34795]: pgmap v739: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-05-17T04:46:20.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:20 smithi110 ceph-mon[35497]: pgmap v739: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-05-17T04:46:20.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:20 smithi050 ceph-mon[31739]: pgmap v739: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-05-17T04:46:22.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:22 smithi105 ceph-mon[34795]: pgmap v740: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.6 MiB/s wr, 284 op/s 2024-05-17T04:46:22.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:22 smithi110 ceph-mon[35497]: pgmap v740: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.6 MiB/s wr, 284 op/s 2024-05-17T04:46:22.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:22 smithi050 ceph-mon[31739]: pgmap v740: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 2.6 MiB/s wr, 284 op/s 2024-05-17T04:46:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 445.0 s, 122.6 tps, lat 824.612 ms stddev 192.270 2024-05-17T04:46:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:24 smithi105 ceph-mon[34795]: pgmap v741: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-05-17T04:46:24.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:24 smithi110 ceph-mon[35497]: pgmap v741: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-05-17T04:46:24.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:24 smithi050 ceph-mon[31739]: pgmap v741: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-05-17T04:46:26.317 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:26 smithi110 ceph-mon[35497]: pgmap v742: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-17T04:46:26.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:26 smithi050 ceph-mon[31739]: pgmap v742: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-17T04:46:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:26 smithi105 ceph-mon[34795]: pgmap v742: 129 pgs: 129 active+clean; 1.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-17T04:46:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.667+0000 7f3c91429640 1 -- 172.21.15.50:0/899224544 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88035020 con 0x7f3c8c1031a0 2024-05-17T04:46:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 -- 172.21.15.50:0/899224544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 msgr2=0x7f3c8c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 --2- 172.21.15.50:0/899224544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c1035b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009900 tx=0x7f3c8802a3f0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 -- 172.21.15.50:0/899224544 shutdown_connections 2024-05-17T04:46:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 --2- 172.21.15.50:0/899224544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c103af0 0x7f3c8c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 --2- 172.21.15.50:0/899224544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c1035b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 --2- 172.21.15.50:0/899224544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c102080 0x7f3c8c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.668+0000 7f3c9468f640 1 -- 172.21.15.50:0/899224544 >> 172.21.15.50:0/899224544 conn(0x7f3c8c0fd880 msgr2=0x7f3c8c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.669+0000 7f3c9468f640 1 -- 172.21.15.50:0/899224544 shutdown_connections 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.669+0000 7f3c9468f640 1 -- 172.21.15.50:0/899224544 wait complete. 2024-05-17T04:46:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 Processor -- start 2024-05-17T04:46:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 -- start start 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 0x7f3c8c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 0x7f3c8c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10c540 con 0x7f3c8c103af0 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10c3c0 con 0x7f3c8c102080 2024-05-17T04:46:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.670+0000 7f3c9468f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10c6c0 con 0x7f3c8c1031a0 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 0x7f3c8c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 0x7f3c8c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42622/0 (socket says 172.21.15.50:42622) 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c9242b640 1 -- 172.21.15.50:0/1634599913 learned_addr learned my addr 172.21.15.50:0/1634599913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1634599913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 msgr2=0x7f3c8c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 0x7f3c8c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 0x7f3c8c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 msgr2=0x7f3c8c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 0x7f3c8c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1634599913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c19f120 con 0x7f3c8c1031a0 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 0x7f3c8c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:46:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.671+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c197b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c8802a910 tx=0x7f3c88005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.672+0000 7f3c7f7fe640 1 -- 172.21.15.50:0/1634599913 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88040070 con 0x7f3c8c1031a0 2024-05-17T04:46:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.672+0000 7f3c7f7fe640 1 -- 172.21.15.50:0/1634599913 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c88004c60 con 0x7f3c8c1031a0 2024-05-17T04:46:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.672+0000 7f3c7f7fe640 1 -- 172.21.15.50:0/1634599913 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88035020 con 0x7f3c8c1031a0 2024-05-17T04:46:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.672+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c19f3b0 con 0x7f3c8c1031a0 2024-05-17T04:46:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.672+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c19f850 con 0x7f3c8c1031a0 2024-05-17T04:46:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.676+0000 7f3c7f7fe640 1 -- 172.21.15.50:0/1634599913 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3c88043070 con 0x7f3c8c1031a0 2024-05-17T04:46:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.677+0000 7f3c7f7fe640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 0x7f3c780b9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.678+0000 7f3c7f7fe640 1 -- 172.21.15.50:0/1634599913 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3c88103e50 con 0x7f3c8c1031a0 2024-05-17T04:46:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.678+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 0x7f3c780b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.688+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 0x7f3c780b9100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c800058c0 tx=0x7f3c80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.689+0000 7f3c9b621740 1 Processor -- start 2024-05-17T04:46:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 -- start start 2024-05-17T04:46:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bd9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555c16b44cc0 con 0x555c1680e4d0 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c16b34860 con 0x555c168062f0 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9b621740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555c16b388a0 con 0x555c1682f1b0 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c91c2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c91c2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42628/0 (socket says 172.21.15.50:42628) 2024-05-17T04:46:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c91c2a640 1 -- 172.21.15.50:0/707127839 learned_addr learned my addr 172.21.15.50:0/707127839 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c92c2c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bd9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.690+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:41218/0 (socket says 172.21.15.50:41218) 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 -- 172.21.15.50:0/707127839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 msgr2=0x555c16bd2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 -- 172.21.15.50:0/707127839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 msgr2=0x555c16bd9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bd9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 -- 172.21.15.50:0/707127839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c16afcd50 con 0x555c1682f1b0 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c9242b640 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009f40 tx=0x7f3c80014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1206b76aa5f6638 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c7cbfe640 1 -- 172.21.15.50:0/707127839 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80003d70 con 0x555c1682f1b0 2024-05-17T04:46:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c7cbfe640 1 -- 172.21.15.50:0/707127839 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8001c880 con 0x555c1682f1b0 2024-05-17T04:46:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.691+0000 7f3c7cbfe640 1 -- 172.21.15.50:0/707127839 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8001d650 con 0x555c1682f1b0 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 -- 172.21.15.50:0/707127839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 msgr2=0x555c16bd2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009f40 tx=0x7f3c80014000 comp rx=0 tx=0).stop 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 -- 172.21.15.50:0/707127839 shutdown_connections 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bd9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bd2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 --2- 172.21.15.50:0/707127839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16bd2770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 -- 172.21.15.50:0/707127839 >> 172.21.15.50:0/707127839 conn(0x555c16823300 msgr2=0x555c16bce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 -- 172.21.15.50:0/707127839 shutdown_connections 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.693+0000 7f3c9b621740 1 -- 172.21.15.50:0/707127839 wait complete. 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.694+0000 7f3c9b621740 1 -- start start 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.694+0000 7f3c9b621740 1 Processor -- start 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bdc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16be34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555c16a70ab0 con 0x555c1680e4d0 2024-05-17T04:46:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c16af14d0 con 0x555c168062f0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555c16bda690 con 0x555c1682f1b0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9b621740 10 client.0 _ll_register_callbacks cb 0x555c16b2f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c91c2a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bdc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42638/0 (socket says 172.21.15.50:42638) 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 -- 172.21.15.50:0/2238103327 learned_addr learned my addr 172.21.15.50:0/2238103327 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16be34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 -- 172.21.15.50:0/2238103327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 msgr2=0x555c16be34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16be34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 -- 172.21.15.50:0/2238103327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 msgr2=0x555c16bdc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bdc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.695+0000 7f3c9242b640 1 -- 172.21.15.50:0/2238103327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c16af2340 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c9242b640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003de0 tx=0x7f3c80003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb01011cb1df4875 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80034060 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8001c880 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3c8001c5b0 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c8003b050 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c9b621740 10 client.25850 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c1689a790 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.696+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8001c880 con 0x555c168062f0 2024-05-17T04:46:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3c8001a070 con 0x555c168062f0 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 10 client.25850 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 10 client.25850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 20 client.25850 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c167edd80 0x555c16be9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 4 client.25850 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.697+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c16b46c10 con 0x555c167edd80 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.698+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c167edd80 0x555c16be9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.698+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c167edd80 0x555c16be9480 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af0f6c857f0af37 server_cookie=4ed919aae2e543e8 in_seq=0 out_seq=0 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.698+0000 7f3c75ffb640 10 client.25850 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c75ffb640 1 -- 172.21.15.50:0/2238103327 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c16b46c10 con 0x555c167edd80 2024-05-17T04:46:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c75ffb640 10 client.25850 handle_command_reply: tid=0 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 client.25850 shutdown 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 msgr2=0x555c16bdc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003de0 tx=0x7f3c80003e10 comp rx=0 tx=0).stop 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 shutdown_connections 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1682f1b0 0x555c16be34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c167edd80 0x555c16be9480 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16bdc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c75ffb640 0 client.25850 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 --2- 172.21.15.50:0/2238103327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bdc040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 >> 172.21.15.50:0/2238103327 conn(0x555c16823300 msgr2=0x555c16bd1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 shutdown_connections 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 1 -- 172.21.15.50:0/2238103327 wait complete. 2024-05-17T04:46:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.699+0000 7f3c9b621740 20 client.25850 trim_cache size 0 max 16384 2024-05-17T04:46:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.727+0000 7f3c9b621740 1 Processor -- start 2024-05-17T04:46:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.727+0000 7f3c9b621740 1 -- start start 2024-05-17T04:46:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c167edd80 0x555c16beeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1680e4d0 0x555c16be3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1ac6a0 con 0x555c167edd80 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1a9f70 con 0x555c168062f0 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9b621740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1a7dc0 con 0x555c1680e4d0 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c91c2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c91c2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42650/0 (socket says 172.21.15.50:42650) 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1521138247 learned_addr learned my addr 172.21.15.50:0/1521138247 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1680e4d0 0x555c16be3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.728+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c167edd80 0x555c16beeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1521138247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1680e4d0 msgr2=0x555c16be3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1680e4d0 0x555c16be3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1521138247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c167edd80 msgr2=0x555c16beeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c167edd80 0x555c16beeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 -- 172.21.15.50:0/1521138247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c16913620 con 0x555c168062f0 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c88101610 tx=0x7f3c88031470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e09426f6bada9c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c77fff640 1 -- 172.21.15.50:0/1521138247 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8804a040 con 0x555c168062f0 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c77fff640 1 -- 172.21.15.50:0/1521138247 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c880d3f40 con 0x555c168062f0 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.729+0000 7f3c77fff640 1 -- 172.21.15.50:0/1521138247 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c880780b0 con 0x555c168062f0 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- 172.21.15.50:0/1521138247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 msgr2=0x555c16bef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c88101610 tx=0x7f3c88031470 comp rx=0 tx=0).stop 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- 172.21.15.50:0/1521138247 shutdown_connections 2024-05-17T04:46:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c1680e4d0 0x555c16be3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c168062f0 0x555c16bef050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1521138247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c167edd80 0x555c16beeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- 172.21.15.50:0/1521138247 >> 172.21.15.50:0/1521138247 conn(0x555c16823300 msgr2=0x555c16bd5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- 172.21.15.50:0/1521138247 shutdown_connections 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- 172.21.15.50:0/1521138247 wait complete. 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.730+0000 7f3c9b621740 1 -- start start 2024-05-17T04:46:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 Processor -- start 2024-05-17T04:46:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c168062f0 0x555c16be2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16be31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1a8430 con 0x555c1680e4d0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1a98a0 con 0x555c167edd80 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c1a8b00 con 0x555c168062f0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9b621740 10 client.0 _ll_register_callbacks cb 0x7f3c84009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9242b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:42654/0 (socket says 172.21.15.50:42654) 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c9242b640 1 -- 172.21.15.50:0/1818509006 learned_addr learned my addr 172.21.15.50:0/1818509006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c168062f0 0x555c16be2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.731+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16be31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 -- 172.21.15.50:0/1818509006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c168062f0 msgr2=0x555c16be2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c168062f0 0x555c16be2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 -- 172.21.15.50:0/1818509006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 msgr2=0x555c16be31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16be31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 -- 172.21.15.50:0/1818509006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c16c4f7e0 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c92c2c640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16be31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9242b640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c80011ca0 tx=0x7f3c80004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b17fc87cb28a6cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80034070 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3c80009e40 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c880d3f40 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3c80004cd0 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9b621740 10 client.25856 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.732+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555c16c50aa0 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c880d3f40 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3c8001a070 con 0x555c167edd80 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 10 client.25856 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 20 client.25856 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c1682f1b0 0x555c16c53510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 4 client.25856 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c16b403f0 con 0x555c1682f1b0 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.733+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c1682f1b0 0x555c16c53510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.734+0000 7f3c75ffb640 10 client.25856 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.734+0000 7f3c91c2a640 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c1682f1b0 0x555c16c53510 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7109f57cc5b94f1 server_cookie=83b8aaa012b384e8 in_seq=0 out_seq=0 2024-05-17T04:46:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.767+0000 7f3c75ffb640 1 -- 172.21.15.50:0/1818509006 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555c16b403f0 con 0x555c1682f1b0 2024-05-17T04:46:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.767+0000 7f3c75ffb640 10 client.25856 handle_command_reply: tid=0 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.767+0000 7f3c9b621740 1 client.25856 shutdown 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.767+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 msgr2=0x555c16be2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.767+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c80011ca0 tx=0x7f3c80004830 comp rx=0 tx=0).stop 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 shutdown_connections 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555c1682f1b0 0x555c16c53510 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555c1680e4d0 0x555c16be31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c168062f0 0x555c16be2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 --2- 172.21.15.50:0/1818509006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c167edd80 0x555c16be2730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c75ffb640 0 client.25856 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 >> 172.21.15.50:0/1818509006 conn(0x555c16823300 msgr2=0x555c16bd6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 shutdown_connections 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 1 -- 172.21.15.50:0/1818509006 wait complete. 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.768+0000 7f3c9b621740 20 client.25856 trim_cache size 0 max 16384 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:46:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 msgr2=0x7f3c780b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 0x7f3c780b9100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c800058c0 tx=0x7f3c80005f90 comp rx=0 tx=0).stop 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 msgr2=0x7f3c8c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c197b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c8802a910 tx=0x7f3c88005df0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 shutdown_connections 2024-05-17T04:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3c780b6bf0 0x7f3c780b9100 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c8c103af0 0x7f3c8c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c8c1031a0 0x7f3c8c197b20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 --2- 172.21.15.50:0/1634599913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c8c102080 0x7f3c8c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 >> 172.21.15.50:0/1634599913 conn(0x7f3c8c0fd880 msgr2=0x7f3c8c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 shutdown_connections 2024-05-17T04:46:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:27.769+0000 7f3c9468f640 1 -- 172.21.15.50:0/1634599913 wait complete. 2024-05-17T04:46:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 450.0 s, 136.2 tps, lat 756.691 ms stddev 233.188 2024-05-17T04:46:28.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:28 smithi050 ceph-mon[31739]: pgmap v743: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-17T04:46:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:28 smithi105 ceph-mon[34795]: pgmap v743: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-17T04:46:28.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:28 smithi110 ceph-mon[35497]: pgmap v743: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-17T04:46:30.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:30 smithi050 ceph-mon[31739]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-17T04:46:30.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:30 smithi105 ceph-mon[34795]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-17T04:46:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:30 smithi110 ceph-mon[35497]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 3.3 MiB/s wr, 334 op/s 2024-05-17T04:46:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:32 smithi105 ceph-mon[34795]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-17T04:46:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:32 smithi110 ceph-mon[35497]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-17T04:46:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:32 smithi050 ceph-mon[31739]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-05-17T04:46:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 455.0 s, 127.0 tps, lat 749.891 ms stddev 202.605 2024-05-17T04:46:34.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:34 smithi105 ceph-mon[34795]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-17T04:46:34.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:34 smithi110 ceph-mon[35497]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-17T04:46:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:34 smithi050 ceph-mon[31739]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.5 MiB/s wr, 337 op/s 2024-05-17T04:46:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:36 smithi105 ceph-mon[34795]: pgmap v747: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-05-17T04:46:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:36 smithi110 ceph-mon[35497]: pgmap v747: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-05-17T04:46:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:36 smithi050 ceph-mon[31739]: pgmap v747: 129 pgs: 129 active+clean; 1.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-05-17T04:46:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 460.0 s, 121.8 tps, lat 813.768 ms stddev 199.673 2024-05-17T04:46:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:38 smithi105 ceph-mon[34795]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-17T04:46:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:38 smithi110 ceph-mon[35497]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-17T04:46:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:38 smithi050 ceph-mon[31739]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-05-17T04:46:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:40 smithi105 ceph-mon[34795]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.5 MiB/s wr, 333 op/s 2024-05-17T04:46:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:40 smithi110 ceph-mon[35497]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.5 MiB/s wr, 333 op/s 2024-05-17T04:46:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:40 smithi050 ceph-mon[31739]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.5 MiB/s wr, 333 op/s 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.892+0000 7fe32f864640 1 -- 172.21.15.50:0/2946810233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 msgr2=0x7fe32810b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.892+0000 7fe32f864640 1 --2- 172.21.15.50:0/2946810233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe32810b500 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe320009ca0 tx=0x7fe32002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.892+0000 7fe32f864640 1 -- 172.21.15.50:0/2946810233 shutdown_connections 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.892+0000 7fe32f864640 1 --2- 172.21.15.50:0/2946810233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe32810b500 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.892+0000 7fe32f864640 1 --2- 172.21.15.50:0/2946810233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 0x7fe3281046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 --2- 172.21.15.50:0/2946810233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe3281035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 -- 172.21.15.50:0/2946810233 >> 172.21.15.50:0/2946810233 conn(0x7fe3280fe970 msgr2=0x7fe328100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 -- 172.21.15.50:0/2946810233 shutdown_connections 2024-05-17T04:46:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 -- 172.21.15.50:0/2946810233 wait complete. 2024-05-17T04:46:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 Processor -- start 2024-05-17T04:46:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.893+0000 7fe32f864640 1 -- start start 2024-05-17T04:46:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 0x7fe328199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe3281a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe32810eb20 con 0x7fe328104c00 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe32810e9a0 con 0x7fe328103190 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32f864640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe32810eca0 con 0x7fe3281042b0 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32de01640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe3281a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44528/0 (socket says 172.21.15.50:44528) 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 0x7fe328199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 -- 172.21.15.50:0/2618869415 learned_addr learned my addr 172.21.15.50:0/2618869415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 -- 172.21.15.50:0/2618869415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 msgr2=0x7fe328199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 0x7fe328199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 -- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 msgr2=0x7fe3281a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe3281a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.894+0000 7fe32d600640 1 -- 172.21.15.50:0/2618869415 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281a12d0 con 0x7fe328103190 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe32d600640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e9c0 tx=0x7fe32400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe31e7fc640 1 -- 172.21.15.50:0/2618869415 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400cd80 con 0x7fe328103190 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe31e7fc640 1 -- 172.21.15.50:0/2618869415 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3240054b0 con 0x7fe328103190 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe31e7fc640 1 -- 172.21.15.50:0/2618869415 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324010430 con 0x7fe328103190 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281a15c0 con 0x7fe328103190 2024-05-17T04:46:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.895+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281a1b00 con 0x7fe328103190 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe31e7fc640 1 -- 172.21.15.50:0/2618869415 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe324004f20 con 0x7fe328103190 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 Processor -- start 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 -- start start 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc0f4d0 0x561edbfdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561edbf45cc0 con 0x561edbc301b0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561edbf35860 con 0x561edbc072f0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe336801740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561edbf398a0 con 0x561edbc0f4d0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe32de01640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe32d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe32de01640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41910/0 (socket says 172.21.15.50:41910) 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe32de01640 1 -- 172.21.15.50:0/219721413 learned_addr learned my addr 172.21.15.50:0/219721413 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.899+0000 7fe32cdff640 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc0f4d0 0x561edbfdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 -- 172.21.15.50:0/219721413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc0f4d0 msgr2=0x561edbfdf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc0f4d0 0x561edbfdf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 -- 172.21.15.50:0/219721413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 msgr2=0x561edbfd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 -- 172.21.15.50:0/219721413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edbefdd50 con 0x561edbc301b0 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe31e7fc640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 0x7fe3140b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32de01640 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe320004ac0 tx=0x7fe320009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c016f1f944946b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe32d600640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 0x7fe3140b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe3137fe640 1 -- 172.21.15.50:0/219721413 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320031a80 con 0x561edbc301b0 2024-05-17T04:46:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe31e7fc640 1 -- 172.21.15.50:0/2618869415 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe3240e2b20 con 0x7fe328103190 2024-05-17T04:46:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe3137fe640 1 -- 172.21.15.50:0/219721413 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe320036070 con 0x561edbc301b0 2024-05-17T04:46:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.900+0000 7fe3137fe640 1 -- 172.21.15.50:0/219721413 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200043a0 con 0x561edbc301b0 2024-05-17T04:46:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 -- 172.21.15.50:0/219721413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 msgr2=0x561edbfd7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe32d600640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 0x7fe3140b4d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe3240e6560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe320004ac0 tx=0x7fe320009800 comp rx=0 tx=0).stop 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 -- 172.21.15.50:0/219721413 shutdown_connections 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc0f4d0 0x561edbfdf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 --2- 172.21.15.50:0/219721413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfd7b80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.901+0000 7fe336801740 1 -- 172.21.15.50:0/219721413 >> 172.21.15.50:0/219721413 conn(0x561edbc24300 msgr2=0x561edbfcf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- 172.21.15.50:0/219721413 shutdown_connections 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- 172.21.15.50:0/219721413 wait complete. 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- start start 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 Processor -- start 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 0x561edbfe8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561edbe71ab0 con 0x561edbc0f4d0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561edbef24d0 con 0x561edbc301b0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561edbfdf8c0 con 0x561edbc072f0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe336801740 10 client.0 _ll_register_callbacks cb 0x561edbf30950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.902+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41912/0 (socket says 172.21.15.50:41912) 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 -- 172.21.15.50:0/1040538407 learned_addr learned my addr 172.21.15.50:0/1040538407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32de01640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32de01640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 0x561edbfe8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 -- 172.21.15.50:0/1040538407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 msgr2=0x561edbfe12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 -- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 msgr2=0x561edbfe8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 0x561edbfe8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 -- 172.21.15.50:0/1040538407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edbef3340 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32de01640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 0x561edbfe8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe32cdff640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe318012720 tx=0x7fe318012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258955f820b502a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe310ff9640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318004030 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe3180041d0 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe320036070 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe318025050 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe336801740 10 client.16110 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.903+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561edbc9b790 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe320036070 con 0x561edbc0f4d0 2024-05-17T04:46:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe318003910 con 0x561edbc0f4d0 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 10 client.16110 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 20 client.16110 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbeed80 0x561edbfeeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 4 client.16110 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561edbf47c10 con 0x561edbbeed80 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe32de01640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbeed80 0x561edbfeeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe310ff9640 10 client.16110 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.904+0000 7fe32de01640 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbeed80 0x561edbfeeb40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b8284283d9166f server_cookie=28968275404441c0 in_seq=0 out_seq=0 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe310ff9640 1 -- 172.21.15.50:0/1040538407 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561edbf47c10 con 0x561edbbeed80 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe310ff9640 10 client.16110 handle_command_reply: tid=0 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 client.16110 shutdown 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 msgr2=0x561edbfe1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe318012720 tx=0x7fe318012be0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 shutdown_connections 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc301b0 0x561edbfe8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbeed80 0x561edbfeeb40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfe1810 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 --2- 172.21.15.50:0/1040538407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe310ff9640 0 client.16110 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.905+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 >> 172.21.15.50:0/1040538407 conn(0x561edbc24300 msgr2=0x561edbfd2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.906+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 shutdown_connections 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.906+0000 7fe336801740 1 -- 172.21.15.50:0/1040538407 wait complete. 2024-05-17T04:46:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.906+0000 7fe336801740 20 client.16110 trim_cache size 0 max 16384 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.925+0000 7fe336801740 1 Processor -- start 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.925+0000 7fe336801740 1 -- start start 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfcdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc301b0 0x561edbfe8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281ac5a0 con 0x561edbc0f4d0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281a9e70 con 0x561edbc072f0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe336801740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281a7cc0 con 0x561edbc301b0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41914/0 (socket says 172.21.15.50:41914) 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 -- 172.21.15.50:0/691588388 learned_addr learned my addr 172.21.15.50:0/691588388 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32de01640 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfcdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32de01640 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc301b0 0x561edbfe8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 -- 172.21.15.50:0/691588388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc301b0 msgr2=0x561edbfe8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc301b0 0x561edbfe8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 -- 172.21.15.50:0/691588388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 msgr2=0x561edbfcdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfcdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.926+0000 7fe32cdff640 1 -- 172.21.15.50:0/691588388 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561edbd14620 con 0x561edbc0f4d0 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.927+0000 7fe32cdff640 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe318012420 tx=0x7fe318009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852bf3d1fc7b44d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.927+0000 7fe312ffd640 1 -- 172.21.15.50:0/691588388 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318022040 con 0x561edbc0f4d0 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.927+0000 7fe312ffd640 1 -- 172.21.15.50:0/691588388 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe318003a80 con 0x561edbc0f4d0 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.927+0000 7fe312ffd640 1 -- 172.21.15.50:0/691588388 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31800fba0 con 0x561edbc0f4d0 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.928+0000 7fe336801740 1 -- 172.21.15.50:0/691588388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 msgr2=0x561edbfce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.928+0000 7fe336801740 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe318012420 tx=0x7fe318009620 comp rx=0 tx=0).stop 2024-05-17T04:46:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 -- 172.21.15.50:0/691588388 shutdown_connections 2024-05-17T04:46:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc301b0 0x561edbfe8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc0f4d0 0x561edbfce430 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 --2- 172.21.15.50:0/691588388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc072f0 0x561edbfcdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 -- 172.21.15.50:0/691588388 >> 172.21.15.50:0/691588388 conn(0x561edbc24300 msgr2=0x561edbfcd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 -- 172.21.15.50:0/691588388 shutdown_connections 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 -- 172.21.15.50:0/691588388 wait complete. 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.929+0000 7fe336801740 1 -- start start 2024-05-17T04:46:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 Processor -- start 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 0x561edbfe6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfe7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281a8330 con 0x561edbc301b0 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281a97a0 con 0x561edbc0f4d0 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3281a8a00 con 0x561edbc072f0 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe32de01640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe32cdff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 0x561edbfe6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe32de01640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48854/0 (socket says 172.21.15.50:48854) 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe336801740 10 client.0 _ll_register_callbacks cb 0x7fe318009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe32de01640 1 -- 172.21.15.50:0/4263782258 learned_addr learned my addr 172.21.15.50:0/4263782258 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.930+0000 7fe32de01640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfe7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 -- 172.21.15.50:0/4263782258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 msgr2=0x561edbfe6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 -- 172.21.15.50:0/4263782258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 msgr2=0x561edbfe7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfe7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 -- 172.21.15.50:0/4263782258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561edbfe7c80 con 0x561edbc0f4d0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32de01640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe32cdff640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 0x561edbfe6760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561edbfe1f70 tx=0x7fe318003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e91c22fec8112e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe310ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:40.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318005280 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe31800f920 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318003a80 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe318038050 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe336801740 10 client.25874 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.931+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561edbfe8f10 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe318003a80 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe318010930 con 0x561edbc0f4d0 2024-05-17T04:46:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 10 client.25874 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 20 client.25874 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbc41c0 0x561edc0592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 4 client.25874 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561edbf413f0 con 0x561edbbc41c0 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.932+0000 7fe32de01640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbc41c0 0x561edc0592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.933+0000 7fe310ff9640 10 client.25874 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.933+0000 7fe32de01640 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbc41c0 0x561edc0592d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a963d6510d8cb1e server_cookie=2840fddae4b11ee5 in_seq=0 out_seq=0 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe310ff9640 1 -- 172.21.15.50:0/4263782258 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561edbf413f0 con 0x561edbbc41c0 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe310ff9640 10 client.25874 handle_command_reply: tid=0 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 client.25874 shutdown 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 msgr2=0x561edbfe6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 0x561edbfe6760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561edbfe1f70 tx=0x7fe318003820 comp rx=0 tx=0).stop 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 shutdown_connections 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561edbbc41c0 0x561edc0592d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561edbc301b0 0x561edbfe7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561edbc0f4d0 0x561edbfe6760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe310ff9640 0 client.25874 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 --2- 172.21.15.50:0/4263782258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561edbc072f0 0x561edbfe6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 >> 172.21.15.50:0/4263782258 conn(0x561edbc24300 msgr2=0x561edbfdafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.969+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 shutdown_connections 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe336801740 1 -- 172.21.15.50:0/4263782258 wait complete. 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe336801740 20 client.25874 trim_cache size 0 max 16384 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:46:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 msgr2=0x7fe3140b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 0x7fe3140b4d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe3240e6560 comp rx=0 tx=0).stop 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 msgr2=0x7fe328199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e9c0 tx=0x7fe32400ee80 comp rx=0 tx=0).stop 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 shutdown_connections 2024-05-17T04:46:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe3140b2860 0x7fe3140b4d70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe328104c00 0x7fe3281a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3281042b0 0x7fe328199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 --2- 172.21.15.50:0/2618869415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe328103190 0x7fe328199790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.970+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 >> 172.21.15.50:0/2618869415 conn(0x7fe3280fe970 msgr2=0x7fe3281075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.971+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 shutdown_connections 2024-05-17T04:46:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:40.971+0000 7fe32f864640 1 -- 172.21.15.50:0/2618869415 wait complete. 2024-05-17T04:46:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:42 smithi105 ceph-mon[34795]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-05-17T04:46:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:42 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:46:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:42 smithi110 ceph-mon[35497]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-05-17T04:46:42.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:42 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:46:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:42 smithi050 ceph-mon[31739]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-05-17T04:46:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:42 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:46:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 465.0 s, 113.4 tps, lat 877.053 ms stddev 205.864 2024-05-17T04:46:43.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:46:43.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:46:43.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:46:43.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:46:43.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:46:43.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:46:43.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:46:43.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:46:43.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:46:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:44 smithi105 ceph-mon[34795]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-05-17T04:46:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:44 smithi110 ceph-mon[35497]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-05-17T04:46:44.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:44 smithi050 ceph-mon[31739]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 3.6 MiB/s wr, 338 op/s 2024-05-17T04:46:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:46 smithi105 ceph-mon[34795]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-17T04:46:46.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:46 smithi110 ceph-mon[35497]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-17T04:46:46.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:46 smithi050 ceph-mon[31739]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.9 MiB/s wr, 282 op/s 2024-05-17T04:46:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 470.0 s, 125.6 tps, lat 817.509 ms stddev 194.004 2024-05-17T04:46:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:48 smithi105 ceph-mon[34795]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-05-17T04:46:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:48 smithi110 ceph-mon[35497]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-05-17T04:46:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:48 smithi050 ceph-mon[31739]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-05-17T04:46:50.378 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:50 smithi050 ceph-mon[31739]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.1 MiB/s wr, 312 op/s 2024-05-17T04:46:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:50 smithi105 ceph-mon[34795]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.1 MiB/s wr, 312 op/s 2024-05-17T04:46:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:50 smithi110 ceph-mon[35497]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.1 MiB/s wr, 312 op/s 2024-05-17T04:46:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:52 smithi105 ceph-mon[34795]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-05-17T04:46:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:52 smithi110 ceph-mon[35497]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-05-17T04:46:52.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:52 smithi050 ceph-mon[31739]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-05-17T04:46:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 475.0 s, 132.0 tps, lat 748.871 ms stddev 330.301 2024-05-17T04:46:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.114+0000 7f104ffff640 1 -- 172.21.15.50:0/2685975619 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050031d80 con 0x7f10581020a0 2024-05-17T04:46:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 -- 172.21.15.50:0/2685975619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10581020a0 msgr2=0x7f10581024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 --2- 172.21.15.50:0/2685975619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10581020a0 0x7f10581024b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1050009ca0 tx=0x7f105002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 -- 172.21.15.50:0/2685975619 shutdown_connections 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 --2- 172.21.15.50:0/2685975619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1058103b10 0x7f105806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 --2- 172.21.15.50:0/2685975619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 0x7f10581035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 --2- 172.21.15.50:0/2685975619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10581020a0 0x7f10581024b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 -- 172.21.15.50:0/2685975619 >> 172.21.15.50:0/2685975619 conn(0x7f10580fd880 msgr2=0x7f10580ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 -- 172.21.15.50:0/2685975619 shutdown_connections 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.115+0000 7f105f88c640 1 -- 172.21.15.50:0/2685975619 wait complete. 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 Processor -- start 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 -- start start 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 0x7f1058105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 0x7f1058104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 0x7f1058104a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f105810eba0 con 0x7f10581020a0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f105810ea20 con 0x7f10581031c0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105f88c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f105810ed20 con 0x7f1058103b10 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.116+0000 7f105d628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 0x7f1058105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105d628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 0x7f1058105e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51006/0 (socket says 172.21.15.50:51006) 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105d628640 1 -- 172.21.15.50:0/1960252413 learned_addr learned my addr 172.21.15.50:0/1960252413 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105ce27640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 0x7f1058104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 0x7f1058104a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 -- 172.21.15.50:0/1960252413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 msgr2=0x7f1058104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 0x7f1058104500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 -- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 msgr2=0x7f1058105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 0x7f1058105e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 -- 172.21.15.50:0/1960252413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105819d6a0 con 0x7f1058103b10 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f105de29640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 0x7f1058104a40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f104800e970 tx=0x7f104800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.117+0000 7f104e7fc640 1 -- 172.21.15.50:0/1960252413 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800ccc0 con 0x7f1058103b10 2024-05-17T04:46:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.118+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10581050a0 con 0x7f1058103b10 2024-05-17T04:46:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.118+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10581a82a0 con 0x7f1058103b10 2024-05-17T04:46:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.118+0000 7f104e7fc640 1 -- 172.21.15.50:0/1960252413 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10480035f0 con 0x7f1058103b10 2024-05-17T04:46:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.118+0000 7f104e7fc640 1 -- 172.21.15.50:0/1960252413 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800ce60 con 0x7f1058103b10 2024-05-17T04:46:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.122+0000 7f104e7fc640 1 -- 172.21.15.50:0/1960252413 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1048004020 con 0x7f1058103b10 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 Processor -- start 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 -- start start 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f104e7fc640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 0x7f10440b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e45d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e45dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f9e3cfcc0 con 0x560f9e0912f0 2024-05-17T04:46:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f9e3bf860 con 0x560f9e0994d0 2024-05-17T04:46:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.123+0000 7f1066817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f9e3c38a0 con 0x560f9e0ba1b0 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105ce27640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 0x7f10440b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:33186/0 (socket says 172.21.15.50:33186) 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 -- 172.21.15.50:0/3171022303 learned_addr learned my addr 172.21.15.50:0/3171022303 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105ce27640 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e45dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105d628640 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e45d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f104e7fc640 1 -- 172.21.15.50:0/1960252413 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f10480e27a0 con 0x7f1058103b10 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 -- 172.21.15.50:0/3171022303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 msgr2=0x560f9e45d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e45d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 -- 172.21.15.50:0/3171022303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 msgr2=0x560f9e45dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e45dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 -- 172.21.15.50:0/3171022303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9e387d50 con 0x560f9e0994d0 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105de29640 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10480a5a50 tx=0x7f10480a59b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45bb95c325dd617e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.124+0000 7f105ce27640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 0x7f10440b4df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1054009730 tx=0x7f1054009b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.125+0000 7f10437fe640 1 -- 172.21.15.50:0/3171022303 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048004310 con 0x560f9e0994d0 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.125+0000 7f10437fe640 1 -- 172.21.15.50:0/3171022303 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10480b2170 con 0x560f9e0994d0 2024-05-17T04:46:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.125+0000 7f10437fe640 1 -- 172.21.15.50:0/3171022303 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048014d10 con 0x560f9e0994d0 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.126+0000 7f1066817740 1 -- 172.21.15.50:0/3171022303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 msgr2=0x560f9e464c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.126+0000 7f1066817740 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10480a5a50 tx=0x7f10480a59b0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.126+0000 7f1066817740 1 -- 172.21.15.50:0/3171022303 shutdown_connections 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e464c10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e45dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 --2- 172.21.15.50:0/3171022303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e45d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 -- 172.21.15.50:0/3171022303 >> 172.21.15.50:0/3171022303 conn(0x560f9e0ae300 msgr2=0x560f9e4591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 -- 172.21.15.50:0/3171022303 shutdown_connections 2024-05-17T04:46:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 -- 172.21.15.50:0/3171022303 wait complete. 2024-05-17T04:46:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 -- start start 2024-05-17T04:46:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.127+0000 7f1066817740 1 Processor -- start 2024-05-17T04:46:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.128+0000 7f1066817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.128+0000 7f1066817740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e467580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.128+0000 7f1066817740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.128+0000 7f105d628640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.128+0000 7f105d628640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33686/0 (socket says 172.21.15.50:33686) 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 -- 172.21.15.50:0/1410282638 learned_addr learned my addr 172.21.15.50:0/1410282638 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f9e2fbab0 con 0x560f9e0ba1b0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f9e37c4d0 con 0x560f9e0994d0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f9e465690 con 0x560f9e0912f0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105de29640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105ce27640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e467580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1066817740 10 client.0 _ll_register_callbacks cb 0x560f9e3ba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 -- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 msgr2=0x560f9e467580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e467580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 -- 172.21.15.50:0/1410282638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 msgr2=0x560f9e46e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9e37d340 con 0x560f9e0912f0 2024-05-17T04:46:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f9e3d1c10 con 0x560f9e0912f0 2024-05-17T04:46:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f105d628640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1050031cb0 tx=0x7f1050031ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f01e4154e814eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.129+0000 7f1040ff9640 10 client.25870 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:46:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.130+0000 7f1040ff9640 1 -- 172.21.15.50:0/1410282638 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003b80 con 0x560f9e0912f0 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.130+0000 7f1040ff9640 1 -- 172.21.15.50:0/1410282638 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1050042070 con 0x560f9e0912f0 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.130+0000 7f1040ff9640 1 -- 172.21.15.50:0/1410282638 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1050003d20 con 0x560f9e0912f0 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.130+0000 7f1066817740 10 client.25870 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.130+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=39}) -- 0x560f9e125790 con 0x560f9e0912f0 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1040ff9640 1 -- 172.21.15.50:0/1410282638 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1050004040 con 0x560f9e0912f0 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 10 client.25870 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 20 client.25870 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e078d80 0x560f9e4746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 4 client.25870 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9e3cb3f0 con 0x560f9e078d80 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.131+0000 7f105ce27640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e078d80 0x560f9e4746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1040ff9640 10 client.25870 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f105ce27640 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e078d80 0x560f9e4746f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4053557b3d03949 server_cookie=a055812299cead20 in_seq=0 out_seq=0 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1040ff9640 1 -- 172.21.15.50:0/1410282638 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f9e3cb3f0 con 0x560f9e078d80 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1040ff9640 10 client.25870 handle_command_reply: tid=0 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1066817740 1 client.25870 shutdown 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 msgr2=0x560f9e467040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.132+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1050031cb0 tx=0x7f1050031ce0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 shutdown_connections 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e078d80 0x560f9e4746f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e467580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 --2- 172.21.15.50:0/1410282638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0912f0 0x560f9e467040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 >> 172.21.15.50:0/1410282638 conn(0x560f9e0ae300 msgr2=0x560f9e4581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1040ff9640 0 client.25870 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 shutdown_connections 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 1 -- 172.21.15.50:0/1410282638 wait complete. 2024-05-17T04:46:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.133+0000 7f1066817740 20 client.25870 trim_cache size 0 max 16384 2024-05-17T04:46:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 Processor -- start 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 -- start start 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0912f0 0x560f9e467420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b4e80 con 0x560f9e0ba1b0 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b2750 con 0x560f9e0912f0 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.161+0000 7f1066817740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b05a0 con 0x560f9e0994d0 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33690/0 (socket says 172.21.15.50:33690) 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 -- 172.21.15.50:0/3780414900 learned_addr learned my addr 172.21.15.50:0/3780414900 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105d628640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0912f0 0x560f9e467420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105de29640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 -- 172.21.15.50:0/3780414900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0912f0 msgr2=0x560f9e467420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0912f0 0x560f9e467420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 -- 172.21.15.50:0/3780414900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 msgr2=0x560f9e46e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 -- 172.21.15.50:0/3780414900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9e19e620 con 0x560f9e0994d0 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105de29640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.162+0000 7f105ce27640 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1058105800 tx=0x7f1054021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb14cf765a30fc8e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.163+0000 7f1042ffd640 1 -- 172.21.15.50:0/3780414900 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054031030 con 0x560f9e0994d0 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.163+0000 7f1042ffd640 1 -- 172.21.15.50:0/3780414900 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f105401fda0 con 0x560f9e0994d0 2024-05-17T04:46:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.163+0000 7f1042ffd640 1 -- 172.21.15.50:0/3780414900 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054005020 con 0x560f9e0994d0 2024-05-17T04:46:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.164+0000 7f1066817740 1 -- 172.21.15.50:0/3780414900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 msgr2=0x560f9e46e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.164+0000 7f1066817740 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1058105800 tx=0x7f1054021000 comp rx=0 tx=0).stop 2024-05-17T04:46:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.164+0000 7f1066817740 1 -- 172.21.15.50:0/3780414900 shutdown_connections 2024-05-17T04:46:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.164+0000 7f1066817740 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0ba1b0 0x560f9e46e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.165+0000 7f1066817740 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0994d0 0x560f9e46e400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.165+0000 7f1066817740 1 --2- 172.21.15.50:0/3780414900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0912f0 0x560f9e467420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.165+0000 7f1066817740 1 -- 172.21.15.50:0/3780414900 >> 172.21.15.50:0/3780414900 conn(0x560f9e0ae300 msgr2=0x560f9e4581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.165+0000 7f1066817740 1 -- 172.21.15.50:0/3780414900 shutdown_connections 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.165+0000 7f1066817740 1 -- 172.21.15.50:0/3780414900 wait complete. 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.166+0000 7f1066817740 1 -- start start 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.166+0000 7f1066817740 1 Processor -- start 2024-05-17T04:46:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f1066817740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f105d628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f105d628640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51054/0 (socket says 172.21.15.50:51054) 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f105d628640 1 -- 172.21.15.50:0/1074948159 learned_addr learned my addr 172.21.15.50:0/1074948159 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e46b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.167+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e46bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b0c10 con 0x560f9e0912f0 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e46b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b2080 con 0x560f9e0994d0 2024-05-17T04:46:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10581b12e0 con 0x560f9e0ba1b0 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105de29640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e46bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1066817740 10 client.0 _ll_register_callbacks cb 0x7f1050009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 -- 172.21.15.50:0/1074948159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 msgr2=0x560f9e46bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e46bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 -- 172.21.15.50:0/1074948159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 msgr2=0x560f9e46eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9e4daa40 con 0x560f9e0994d0 2024-05-17T04:46:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105d628640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f105401fda0 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f105ce27640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e46b620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1058105800 tx=0x7f1054011100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4db0ee9002f4f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.168+0000 7f1040ff9640 10 client.25892 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1040ff9640 1 -- 172.21.15.50:0/1074948159 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054031040 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1040ff9640 1 -- 172.21.15.50:0/1074948159 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f105401a070 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1040ff9640 1 -- 172.21.15.50:0/1074948159 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f105400fdc0 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1066817740 10 client.25892 fetch_fsmap learned FSMap version 39 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=39}) -- 0x560f9e4dba60 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.169+0000 7f1040ff9640 1 -- 172.21.15.50:0/1074948159 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f105401f910 con 0x560f9e0994d0 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 10 client.25892 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:46:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 20 client.25892 populate_metadata read hostname 'smithi050' 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e04e1c0 0x560f9e4de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 4 client.25892 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f9e345750 con 0x560f9e04e1c0 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.170+0000 7f105d628640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e04e1c0 0x560f9e4de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.171+0000 7f105d628640 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e04e1c0 0x560f9e4de2c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11dffc28c903f2f server_cookie=16ec0b40e9fddcf4 in_seq=0 out_seq=0 2024-05-17T04:46:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.171+0000 7f1040ff9640 10 client.25892 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1040ff9640 1 -- 172.21.15.50:0/1074948159 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560f9e345750 con 0x560f9e04e1c0 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1040ff9640 10 client.25892 handle_command_reply: tid=0 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1066817740 1 client.25892 shutdown 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 msgr2=0x560f9e46b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e46b620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1058105800 tx=0x7f1054011100 comp rx=0 tx=0).stop 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.202+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 shutdown_connections 2024-05-17T04:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f9e0ba1b0 0x560f9e46bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f9e0994d0 0x560f9e46b620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x560f9e04e1c0 0x560f9e4de2c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 --2- 172.21.15.50:0/1074948159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f9e0912f0 0x560f9e46eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1040ff9640 0 client.25892 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 >> 172.21.15.50:0/1074948159 conn(0x560f9e0ae300 msgr2=0x560f9e467420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 shutdown_connections 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 1 -- 172.21.15.50:0/1074948159 wait complete. 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.203+0000 7f1066817740 20 client.25892 trim_cache size 0 max 16384 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:46:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:46:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:46:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 msgr2=0x7f10440b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 0x7f10440b4df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1054009730 tx=0x7f1054009b10 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 msgr2=0x7f1058104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 0x7f1058104a40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f104800e970 tx=0x7f104800ee30 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 shutdown_connections 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1058103b10 0x7f1058104a40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10581031c0 0x7f1058104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f10440b28e0 0x7f10440b4df0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 --2- 172.21.15.50:0/1960252413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10581020a0 0x7f1058105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 >> 172.21.15.50:0/1960252413 conn(0x7f10580fd880 msgr2=0x7f1058108600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 shutdown_connections 2024-05-17T04:46:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:46:54.204+0000 7f105f88c640 1 -- 172.21.15.50:0/1960252413 wait complete. 2024-05-17T04:46:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:54 smithi105 ceph-mon[34795]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:46:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:54 smithi110 ceph-mon[35497]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:46:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:54 smithi050 ceph-mon[31739]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:46:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:56 smithi105 ceph-mon[34795]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-05-17T04:46:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:56 smithi110 ceph-mon[35497]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-05-17T04:46:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:56 smithi050 ceph-mon[31739]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-05-17T04:46:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 480.0 s, 152.8 tps, lat 655.476 ms stddev 232.423 2024-05-17T04:46:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:46:58 smithi105 ceph-mon[34795]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-17T04:46:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:46:58 smithi110 ceph-mon[35497]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-17T04:46:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:46:58 smithi050 ceph-mon[31739]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-05-17T04:47:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:00 smithi105 ceph-mon[34795]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.1 MiB/s wr, 286 op/s 2024-05-17T04:47:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:00 smithi110 ceph-mon[35497]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.1 MiB/s wr, 286 op/s 2024-05-17T04:47:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:00 smithi050 ceph-mon[31739]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 3.1 MiB/s wr, 286 op/s 2024-05-17T04:47:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:02 smithi105 ceph-mon[34795]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:47:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:02 smithi110 ceph-mon[35497]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:47:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:02 smithi050 ceph-mon[31739]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-05-17T04:47:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 485.0 s, 128.6 tps, lat 787.498 ms stddev 190.281 2024-05-17T04:47:04.367 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:04 smithi105 ceph-mon[34795]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-05-17T04:47:04.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:04 smithi110 ceph-mon[35497]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-05-17T04:47:04.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:04 smithi050 ceph-mon[31739]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-05-17T04:47:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:06 smithi105 ceph-mon[34795]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 344 op/s 2024-05-17T04:47:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:06 smithi110 ceph-mon[35497]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 344 op/s 2024-05-17T04:47:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:06 smithi050 ceph-mon[31739]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 3.6 MiB/s wr, 344 op/s 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 -- 172.21.15.50:0/2963814023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 msgr2=0x7f927c1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 --2- 172.21.15.50:0/2963814023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1046d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9270009860 tx=0x7f927002a340 comp rx=0 tx=0).stop 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 -- 172.21.15.50:0/2963814023 shutdown_connections 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 --2- 172.21.15.50:0/2963814023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c104c10 0x7f927c10b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 --2- 172.21.15.50:0/2963814023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1046d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 --2- 172.21.15.50:0/2963814023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1031a0 0x7f927c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 -- 172.21.15.50:0/2963814023 >> 172.21.15.50:0/2963814023 conn(0x7f927c0fe9a0 msgr2=0x7f927c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 -- 172.21.15.50:0/2963814023 shutdown_connections 2024-05-17T04:47:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.345+0000 7f928378c640 1 -- 172.21.15.50:0/2963814023 wait complete. 2024-05-17T04:47:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.346+0000 7f928378c640 1 Processor -- start 2024-05-17T04:47:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.346+0000 7f928378c640 1 -- start start 2024-05-17T04:47:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.346+0000 7f928378c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c1031a0 0x7f927c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f928378c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f9281528640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c1031a0 0x7f927c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f9280d27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f9280d27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:55498/0 (socket says 172.21.15.50:55498) 2024-05-17T04:47:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f9280d27640 1 -- 172.21.15.50:0/3094508452 learned_addr learned my addr 172.21.15.50:0/3094508452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c104c10 0x7f927c1a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0761d0 con 0x7f927c104c10 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c076050 con 0x7f927c1042c0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f927c076350 con 0x7f927c1031a0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.347+0000 7f9281d29640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c104c10 0x7f927c1a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 -- 172.21.15.50:0/3094508452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c1031a0 msgr2=0x7f927c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c1031a0 0x7f927c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 -- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c104c10 msgr2=0x7f927c1a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c104c10 0x7f927c1a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1a7960 con 0x7f927c1042c0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f9280d27640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f927002a930 tx=0x7f927002ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f926e7fc640 1 -- 172.21.15.50:0/3094508452 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270040030 con 0x7f927c1042c0 2024-05-17T04:47:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f926e7fc640 1 -- 172.21.15.50:0/3094508452 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9270032bc0 con 0x7f927c1042c0 2024-05-17T04:47:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.348+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c06c760 con 0x7f927c1042c0 2024-05-17T04:47:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.349+0000 7f926e7fc640 1 -- 172.21.15.50:0/3094508452 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270037040 con 0x7f927c1042c0 2024-05-17T04:47:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.349+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c06cbe0 con 0x7f927c1042c0 2024-05-17T04:47:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.352+0000 7f928a649740 1 Processor -- start 2024-05-17T04:47:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.352+0000 7f928a649740 1 -- start start 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f926e7fc640 1 -- 172.21.15.50:0/3094508452 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9270043070 con 0x7f927c1042c0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d074d0 0x5612660d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56126603dcc0 con 0x561265cff2f0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56126602d860 con 0x561265d074d0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f928a649740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5612660318a0 con 0x561265d281b0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9280d27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d074d0 0x5612660d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:37870/0 (socket says 172.21.15.50:37870) 2024-05-17T04:47:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 -- 172.21.15.50:0/1029344380 learned_addr learned my addr 172.21.15.50:0/1029344380 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 -- 172.21.15.50:0/1029344380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d074d0 msgr2=0x5612660d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d074d0 0x5612660d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.353+0000 7f9281d29640 1 -- 172.21.15.50:0/1029344380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 msgr2=0x5612660cbcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281528640 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281d29640 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281d29640 1 -- 172.21.15.50:0/1029344380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561265ff5d50 con 0x561265d281b0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281528640 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f926e7fc640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 0x7f92680b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281528640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 0x7f92680b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.354+0000 7f9281d29640 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9274009fd0 tx=0x7f92740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592a2157d53f0d7f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.355+0000 7f926e7fc640 1 -- 172.21.15.50:0/3094508452 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9270102360 con 0x7f927c1042c0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.355+0000 7f92677fe640 1 -- 172.21.15.50:0/1029344380 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004d20 con 0x561265d281b0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.355+0000 7f92677fe640 1 -- 172.21.15.50:0/1029344380 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9274004050 con 0x561265d281b0 2024-05-17T04:47:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.355+0000 7f92677fe640 1 -- 172.21.15.50:0/1029344380 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274005260 con 0x561265d281b0 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- 172.21.15.50:0/1029344380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 msgr2=0x5612660cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9274009fd0 tx=0x7f92740118d0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f9281528640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 0x7f92680b4d70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5612660ccb70 tx=0x7f9278009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- 172.21.15.50:0/1029344380 shutdown_connections 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d074d0 0x5612660d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 --2- 172.21.15.50:0/1029344380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d281b0 0x5612660cb770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- 172.21.15.50:0/1029344380 >> 172.21.15.50:0/1029344380 conn(0x561265d1c300 msgr2=0x5612660c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- 172.21.15.50:0/1029344380 shutdown_connections 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- 172.21.15.50:0/1029344380 wait complete. 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.356+0000 7f928a649740 1 -- start start 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 Processor -- start 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 0x5612660d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265d281b0 0x5612660dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561265f69ab0 con 0x561265d281b0 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561265fea4d0 con 0x561265cff2f0 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5612660d3690 con 0x561265d074d0 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f9281528640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f928a649740 10 client.0 _ll_register_callbacks cb 0x561266028950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f9281528640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:55524/0 (socket says 172.21.15.50:55524) 2024-05-17T04:47:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f9281528640 1 -- 172.21.15.50:0/2981429025 learned_addr learned my addr 172.21.15.50:0/2981429025 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f9280d27640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 0x5612660d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.357+0000 7f9281d29640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265d281b0 0x5612660dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 -- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 msgr2=0x5612660d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 -- 172.21.15.50:0/2981429025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265d281b0 msgr2=0x5612660dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265d281b0 0x5612660dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 -- 172.21.15.50:0/2981429025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561265feb340 con 0x561265d074d0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9281528640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9280d27640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 0x5612660d5660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92700ffbc0 tx=0x7f92700c94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549e787e0d78abd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700c8e90 con 0x561265d074d0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9274004050 con 0x561265d074d0 2024-05-17T04:47:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f92700b1110 con 0x561265d074d0 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9270032d70 con 0x561265d074d0 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.358+0000 7f928a649740 10 client.25888 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561265d93790 con 0x561265d074d0 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9274004050 con 0x561265d074d0 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f927004d060 con 0x561265d074d0 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 10 client.25888 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 20 client.25888 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561265ce6d80 0x5612660e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 4 client.25888 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56126603fc10 con 0x561265ce6d80 2024-05-17T04:47:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.359+0000 7f9281528640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561265ce6d80 0x5612660e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.360+0000 7f9264ff9640 10 client.25888 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.360+0000 7f9281528640 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561265ce6d80 0x5612660e2810 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b662e1af521e508 server_cookie=72c9f95f88ffb218 in_seq=0 out_seq=0 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f9264ff9640 1 -- 172.21.15.50:0/2981429025 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56126603fc10 con 0x561265ce6d80 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f9264ff9640 10 client.25888 handle_command_reply: tid=0 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 client.25888 shutdown 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 msgr2=0x5612660d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 0x5612660d5660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92700ffbc0 tx=0x7f92700c94f0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 shutdown_connections 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561265ce6d80 0x5612660e2810 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265d281b0 0x5612660dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265d074d0 0x5612660d5660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 --2- 172.21.15.50:0/2981429025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265cff2f0 0x5612660d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 >> 172.21.15.50:0/2981429025 conn(0x561265d1c300 msgr2=0x5612660ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f9264ff9640 0 client.25888 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 shutdown_connections 2024-05-17T04:47:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 1 -- 172.21.15.50:0/2981429025 wait complete. 2024-05-17T04:47:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.361+0000 7f928a649740 20 client.25888 trim_cache size 0 max 16384 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.384+0000 7f928a649740 1 Processor -- start 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 -- start start 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cbc1c0 0x5612660e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1ac570 con 0x561265cff2f0 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1a9e40 con 0x561265d281b0 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f928a649740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0747d0 con 0x561265cbc1c0 2024-05-17T04:47:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9281d29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cbc1c0 0x5612660e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:55534/0 (socket says 172.21.15.50:55534) 2024-05-17T04:47:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9281528640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 -- 172.21.15.50:0/984136492 learned_addr learned my addr 172.21.15.50:0/984136492 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 -- 172.21.15.50:0/984136492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cbc1c0 msgr2=0x5612660e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cbc1c0 0x5612660e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 -- 172.21.15.50:0/984136492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 msgr2=0x5612660cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.385+0000 7f9280d27640 1 -- 172.21.15.50:0/984136492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561265e0c620 con 0x561265d281b0 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.386+0000 7f9281528640 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.386+0000 7f9280d27640 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92700d01a0 tx=0x7f92700fff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb69f20e8380c4d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.388 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.386+0000 7f9266ffd640 1 -- 172.21.15.50:0/984136492 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700c8e90 con 0x561265d281b0 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.386+0000 7f9266ffd640 1 -- 172.21.15.50:0/984136492 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92700c9230 con 0x561265d281b0 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.386+0000 7f9266ffd640 1 -- 172.21.15.50:0/984136492 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700d2060 con 0x561265d281b0 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 -- 172.21.15.50:0/984136492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 msgr2=0x5612660dc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92700d01a0 tx=0x7f92700fff90 comp rx=0 tx=0).stop 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 -- 172.21.15.50:0/984136492 shutdown_connections 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cbc1c0 0x5612660e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x5612660dc510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 --2- 172.21.15.50:0/984136492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cff2f0 0x5612660cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 -- 172.21.15.50:0/984136492 >> 172.21.15.50:0/984136492 conn(0x561265d1c300 msgr2=0x5612660ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 -- 172.21.15.50:0/984136492 shutdown_connections 2024-05-17T04:47:07.389 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.387+0000 7f928a649740 1 -- 172.21.15.50:0/984136492 wait complete. 2024-05-17T04:47:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f928a649740 1 -- start start 2024-05-17T04:47:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f928a649740 1 Processor -- start 2024-05-17T04:47:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f928a649740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 0x5612660db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f928a649740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cff2f0 0x5612660dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f9281528640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 0x5612660db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.388+0000 7f9281528640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 0x5612660db610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52422/0 (socket says 172.21.15.50:52422) 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281528640 1 -- 172.21.15.50:0/2885193965 learned_addr learned my addr 172.21.15.50:0/2885193965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9280d27640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cff2f0 0x5612660dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x561266148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1a8440 con 0x561265cbc1c0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1a9770 con 0x561265d281b0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1a89d0 con 0x561265cff2f0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f928a649740 10 client.0 _ll_register_callbacks cb 0x7f9270005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x561266148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 -- 172.21.15.50:0/2885193965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cff2f0 msgr2=0x5612660dbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cff2f0 0x5612660dbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 -- 172.21.15.50:0/2885193965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 msgr2=0x5612660db610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 0x5612660db610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.389+0000 7f9281d29640 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561266149200 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9281d29640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x561266148bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f927401b040 tx=0x7f9274004b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b66ee5b77ed91b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004050 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f92740041f0 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92700c9230 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f92740043f0 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f928a649740 10 client.25910 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56126614a430 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.390+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92700c9230 con 0x561265d281b0 2024-05-17T04:47:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9274005840 con 0x561265d281b0 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 10 client.25910 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 20 client.25910 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56126614abf0 0x56126614d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 4 client.25910 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612660393f0 con 0x56126614abf0 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f9281528640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56126614abf0 0x56126614d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.391+0000 7f9264ff9640 10 client.25910 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.392+0000 7f9281528640 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56126614abf0 0x56126614d010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc16644473e030b server_cookie=2cbf9421f82ee907 in_seq=0 out_seq=0 2024-05-17T04:47:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f9264ff9640 1 -- 172.21.15.50:0/2885193965 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5612660393f0 con 0x56126614abf0 2024-05-17T04:47:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f9264ff9640 10 client.25910 handle_command_reply: tid=0 2024-05-17T04:47:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 client.25910 shutdown 2024-05-17T04:47:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 msgr2=0x561266148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x561266148bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f927401b040 tx=0x7f9274004b00 comp rx=0 tx=0).stop 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 shutdown_connections 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561265d281b0 0x561266148bc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561265cff2f0 0x5612660dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56126614abf0 0x56126614d010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f9264ff9640 0 client.25910 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 --2- 172.21.15.50:0/2885193965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561265cbc1c0 0x5612660db610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 >> 172.21.15.50:0/2885193965 conn(0x561265d1c300 msgr2=0x5612660cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 shutdown_connections 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 1 -- 172.21.15.50:0/2885193965 wait complete. 2024-05-17T04:47:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.423+0000 7f928a649740 20 client.25910 trim_cache size 0 max 16384 2024-05-17T04:47:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:47:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:47:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:47:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 msgr2=0x7f92680b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 0x7f92680b4d70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5612660ccb70 tx=0x7f9278009260 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 msgr2=0x7f927c1a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f927002a930 tx=0x7f927002ad70 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 shutdown_connections 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f92680b2860 0x7f92680b4d70 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5612660ccb70 tx=0x7f9278009260 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c104c10 0x7f927c1a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c1042c0 0x7f927c1a0360 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f927002a930 tx=0x7f927002ad70 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 --2- 172.21.15.50:0/3094508452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f927c1031a0 0x7f927c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 >> 172.21.15.50:0/3094508452 conn(0x7f927c0fe9a0 msgr2=0x7f927c10f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 shutdown_connections 2024-05-17T04:47:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:07.425+0000 7f928378c640 1 -- 172.21.15.50:0/3094508452 wait complete. 2024-05-17T04:47:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 490.0 s, 127.4 tps, lat 776.984 ms stddev 181.992 2024-05-17T04:47:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:08 smithi105 ceph-mon[34795]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-05-17T04:47:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:08 smithi110 ceph-mon[35497]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-05-17T04:47:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:08 smithi050 ceph-mon[31739]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.2 MiB/s wr, 295 op/s 2024-05-17T04:47:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:10 smithi105 ceph-mon[34795]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-17T04:47:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:10 smithi110 ceph-mon[35497]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-17T04:47:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:10 smithi050 ceph-mon[31739]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.2 MiB/s wr, 296 op/s 2024-05-17T04:47:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:12 smithi105 ceph-mon[34795]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-17T04:47:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:12 smithi110 ceph-mon[35497]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-17T04:47:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:12 smithi050 ceph-mon[31739]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-05-17T04:47:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 495.0 s, 131.4 tps, lat 763.833 ms stddev 209.245 2024-05-17T04:47:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:14 smithi105 ceph-mon[34795]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-05-17T04:47:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:14 smithi110 ceph-mon[35497]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-05-17T04:47:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:14 smithi050 ceph-mon[31739]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-05-17T04:47:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:16 smithi105 ceph-mon[34795]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-05-17T04:47:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:16 smithi110 ceph-mon[35497]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-05-17T04:47:16.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:16 smithi050 ceph-mon[31739]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-05-17T04:47:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 500.0 s, 129.6 tps, lat 765.481 ms stddev 170.322 2024-05-17T04:47:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:18 smithi105 ceph-mon[34795]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2024-05-17T04:47:18.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:18 smithi110 ceph-mon[35497]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2024-05-17T04:47:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:18 smithi050 ceph-mon[31739]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.8 MiB/s wr, 278 op/s 2024-05-17T04:47:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:20 smithi105 ceph-mon[34795]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-05-17T04:47:20.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:20 smithi110 ceph-mon[35497]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 -- 172.21.15.50:0/1451102027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 msgr2=0x7f4db0110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/1451102027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db0110050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009860 tx=0x7f4d9802a340 comp rx=0 tx=0).stop 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 -- 172.21.15.50:0/1451102027 shutdown_connections 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/1451102027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/1451102027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db0110050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/1451102027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db010d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.569+0000 7f4db4fe8640 1 -- 172.21.15.50:0/1451102027 >> 172.21.15.50:0/1451102027 conn(0x7f4db00705e0 msgr2=0x7f4db0072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.570+0000 7f4db4fe8640 1 -- 172.21.15.50:0/1451102027 shutdown_connections 2024-05-17T04:47:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.570+0000 7f4db4fe8640 1 -- 172.21.15.50:0/1451102027 wait complete. 2024-05-17T04:47:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.571+0000 7f4db4fe8640 1 Processor -- start 2024-05-17T04:47:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.571+0000 7f4db4fe8640 1 -- start start 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.571+0000 7f4db4fe8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db01a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4db4fe8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db01a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4db4fe8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4db4fe8640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01162a0 con 0x7f4db0109f20 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4db4fe8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0116120 con 0x7f4db010dc30 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4db4fe8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0116420 con 0x7f4db0110590 2024-05-17T04:47:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db01a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db01a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49382/0 (socket says 172.21.15.50:49382) 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4da7fff640 1 -- 172.21.15.50:0/296331627 learned_addr learned my addr 172.21.15.50:0/296331627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db01a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4daed9d640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 -- 172.21.15.50:0/296331627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 msgr2=0x7f4db01ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 -- 172.21.15.50:0/296331627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 msgr2=0x7f4db01a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db01a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4dae59c640 1 -- 172.21.15.50:0/296331627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db01abba0 con 0x7f4db0109f20 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.572+0000 7f4daed9d640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:47:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4dae59c640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db01a4060 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4db0074880 tx=0x7f4da000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4da77fe640 1 -- 172.21.15.50:0/296331627 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da000cc40 con 0x7f4db0109f20 2024-05-17T04:47:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4da77fe640 1 -- 172.21.15.50:0/296331627 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da000cde0 con 0x7f4db0109f20 2024-05-17T04:47:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4da77fe640 1 -- 172.21.15.50:0/296331627 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00186f0 con 0x7f4db0109f20 2024-05-17T04:47:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01abe90 con 0x7f4db0109f20 2024-05-17T04:47:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.573+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01ac2a0 con 0x7f4db0109f20 2024-05-17T04:47:20.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:20 smithi050 ceph-mon[31739]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 Processor -- start 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 -- start start 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4da77fe640 1 -- 172.21.15.50:0/296331627 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4da0010a70 con 0x7f4db0109f20 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b7d1b0 0x559015f20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f20cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f27c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559015e92cc0 con 0x559015b542f0 2024-05-17T04:47:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559015e82860 con 0x559015b5c4d0 2024-05-17T04:47:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.579+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559015e868a0 con 0x559015b7d1b0 2024-05-17T04:47:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f20cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f20cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53130/0 (socket says 172.21.15.50:53130) 2024-05-17T04:47:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4dae59c640 1 -- 172.21.15.50:0/354477943 learned_addr learned my addr 172.21.15.50:0/354477943 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4da7fff640 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f27c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4daed9d640 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b7d1b0 0x559015f20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4da7fff640 1 -- 172.21.15.50:0/354477943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b7d1b0 msgr2=0x559015f20770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4da7fff640 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b7d1b0 0x559015f20770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4da7fff640 1 -- 172.21.15.50:0/354477943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 msgr2=0x559015f20cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.580+0000 7f4da7fff640 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f20cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.581+0000 7f4da7fff640 1 -- 172.21.15.50:0/354477943 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559015e4ad50 con 0x559015b5c4d0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.581+0000 7f4da7fff640 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f27c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009830 tx=0x7f4d98005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aef6216a8653177 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.581+0000 7f4da4ff9640 1 -- 172.21.15.50:0/354477943 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98041030 con 0x559015b5c4d0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.581+0000 7f4da4ff9640 1 -- 172.21.15.50:0/354477943 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d98004040 con 0x559015b5c4d0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.581+0000 7f4da4ff9640 1 -- 172.21.15.50:0/354477943 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98037040 con 0x559015b5c4d0 2024-05-17T04:47:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.582+0000 7f4da77fe640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 0x7f4d9c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.582+0000 7f4da77fe640 1 -- 172.21.15.50:0/296331627 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4da00e2770 con 0x7f4db0109f20 2024-05-17T04:47:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.582+0000 7f4dbbf76740 1 -- 172.21.15.50:0/354477943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 msgr2=0x559015f27c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.582+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f27c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009830 tx=0x7f4d98005c90 comp rx=0 tx=0).stop 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 -- 172.21.15.50:0/354477943 shutdown_connections 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f27c10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f20cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/354477943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b7d1b0 0x559015f20770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 -- 172.21.15.50:0/354477943 >> 172.21.15.50:0/354477943 conn(0x559015b71300 msgr2=0x559015f1c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4daed9d640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 0x7f4d9c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 -- 172.21.15.50:0/354477943 shutdown_connections 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.583+0000 7f4dbbf76740 1 -- 172.21.15.50:0/354477943 wait complete. 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.584+0000 7f4daed9d640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 0x7f4d9c0b4e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559015f21960 tx=0x7f4da80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.584+0000 7f4dbbf76740 1 -- start start 2024-05-17T04:47:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.584+0000 7f4dbbf76740 1 Processor -- start 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:35862/0 (socket says 172.21.15.50:35862) 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49400/0 (socket says 172.21.15.50:49400) 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dae59c640 1 -- 172.21.15.50:0/1333057464 learned_addr learned my addr 172.21.15.50:0/1333057464 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b7d1b0 0x559015f39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559015dbeab0 con 0x559015b7d1b0 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559015e3f4d0 con 0x559015b5c4d0 2024-05-17T04:47:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.585+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559015f30e40 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dbbf76740 10 client.0 _ll_register_callbacks cb 0x559015e7d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4daed9d640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b7d1b0 0x559015f39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 -- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 msgr2=0x559015f32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f32cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 -- 172.21.15.50:0/1333057464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b7d1b0 msgr2=0x559015f39c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b7d1b0 0x559015f39c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559015e40340 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dae59c640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4da00dff30 tx=0x7f4da0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3116275d068cad88 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00a9220 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4da00a60a0 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.586+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d98004040 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4da00a5240 con 0x559015b542f0 2024-05-17T04:47:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 10 client.25906 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559015be8790 con 0x559015b542f0 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da00bee50 con 0x559015b542f0 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4da00c5020 con 0x559015b542f0 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 10 client.25906 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 20 client.25906 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b3bd80 0x559015f40060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 4 client.25906 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559015e94c10 con 0x559015b3bd80 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.587+0000 7f4da7fff640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b3bd80 0x559015f40060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.588+0000 7f4d95ffb640 10 client.25906 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.588+0000 7f4da7fff640 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b3bd80 0x559015f40060 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af88e0e148a726ee server_cookie=4cc0fb15f0462bb0 in_seq=0 out_seq=0 2024-05-17T04:47:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4d95ffb640 1 -- 172.21.15.50:0/1333057464 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559015e94c10 con 0x559015b3bd80 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4d95ffb640 10 client.25906 handle_command_reply: tid=0 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 client.25906 shutdown 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 msgr2=0x559015f327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4da00dff30 tx=0x7f4da0009700 comp rx=0 tx=0).stop 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 shutdown_connections 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b3bd80 0x559015f40060 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b7d1b0 0x559015f39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b5c4d0 0x559015f32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4d95ffb640 0 client.25906 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/1333057464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f327b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 >> 172.21.15.50:0/1333057464 conn(0x559015b71300 msgr2=0x559015f1f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 shutdown_connections 2024-05-17T04:47:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 1 -- 172.21.15.50:0/1333057464 wait complete. 2024-05-17T04:47:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.589+0000 7f4dbbf76740 20 client.25906 trim_cache size 0 max 16384 2024-05-17T04:47:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 Processor -- start 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 -- start start 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f1f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015f39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b4e10 con 0x559015b5c4d0 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b26e0 con 0x559015b7d1b0 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b0550 con 0x559015b542f0 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53144/0 (socket says 172.21.15.50:53144) 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4da7fff640 1 -- 172.21.15.50:0/646342440 learned_addr learned my addr 172.21.15.50:0/646342440 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4daed9d640 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015f39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.608+0000 7f4dae59c640 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f1f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 -- 172.21.15.50:0/646342440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 msgr2=0x559015f1f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f1f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 -- 172.21.15.50:0/646342440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 msgr2=0x559015f39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015f39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 -- 172.21.15.50:0/646342440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559015c61620 con 0x559015b5c4d0 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4da7fff640 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a930 tx=0x7f4d98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84379eacae74921 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4d97fff640 1 -- 172.21.15.50:0/646342440 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98041040 con 0x559015b5c4d0 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4d97fff640 1 -- 172.21.15.50:0/646342440 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d9800b040 con 0x559015b5c4d0 2024-05-17T04:47:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.609+0000 7f4d97fff640 1 -- 172.21.15.50:0/646342440 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98037040 con 0x559015b5c4d0 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 -- 172.21.15.50:0/646342440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 msgr2=0x559015f1fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4d9802a930 tx=0x7f4d98000bc0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 -- 172.21.15.50:0/646342440 shutdown_connections 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015f39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b5c4d0 0x559015f1fb20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/646342440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b542f0 0x559015f1f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 -- 172.21.15.50:0/646342440 >> 172.21.15.50:0/646342440 conn(0x559015b71300 msgr2=0x559015f1eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 -- 172.21.15.50:0/646342440 shutdown_connections 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.610+0000 7f4dbbf76740 1 -- 172.21.15.50:0/646342440 wait complete. 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 -- start start 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 Processor -- start 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 0x559015f37770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015fa63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b0ba0 con 0x559015b542f0 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b2010 con 0x559015b7d1b0 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b1270 con 0x559015b5c4d0 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dbbf76740 10 client.0 _ll_register_callbacks cb 0x7f4da00a5970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 0x559015f37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53146/0 (socket says 172.21.15.50:53146) 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 -- 172.21.15.50:0/2390822094 learned_addr learned my addr 172.21.15.50:0/2390822094 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4daed9d640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015fa63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 -- 172.21.15.50:0/2390822094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 msgr2=0x559015f37770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 0x559015f37770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 -- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 msgr2=0x559015fa63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015fa63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4dae59c640 1 -- 172.21.15.50:0/2390822094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559015f3a710 con 0x559015b542f0 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.611+0000 7f4da7fff640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 0x559015f37770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4dae59c640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4da00e0250 tx=0x7f4da000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd98b434725ecaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00a9220 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4da00a60a0 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9800b040 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4da00a95c0 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4dbbf76740 10 client.16167 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559015fa7950 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d9800b040 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.612+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4da00c5040 con 0x559015b542f0 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 10 client.16167 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 20 client.16167 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b111c0 0x559015faa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 4 client.16167 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:47:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559015e8e3f0 con 0x559015b111c0 2024-05-17T04:47:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4da7fff640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b111c0 0x559015faa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4d95ffb640 10 client.16167 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.613+0000 7f4da7fff640 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b111c0 0x559015faa1b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1e6f2393aeaca0 server_cookie=f06fd72fbb754afe in_seq=0 out_seq=0 2024-05-17T04:47:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4d95ffb640 1 -- 172.21.15.50:0/2390822094 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559015e8e3f0 con 0x559015b111c0 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4d95ffb640 10 client.16167 handle_command_reply: tid=0 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 client.16167 shutdown 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 msgr2=0x559015f37230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4da00e0250 tx=0x7f4da000cf90 comp rx=0 tx=0).stop 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 shutdown_connections 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559015b7d1b0 0x559015fa63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559015b5c4d0 0x559015f37770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x559015b111c0 0x559015faa1b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 --2- 172.21.15.50:0/2390822094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559015b542f0 0x559015f37230 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4d95ffb640 0 client.16167 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 >> 172.21.15.50:0/2390822094 conn(0x559015b71300 msgr2=0x559015f1f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 shutdown_connections 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 1 -- 172.21.15.50:0/2390822094 wait complete. 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.651+0000 7f4dbbf76740 20 client.16167 trim_cache size 0 max 16384 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:47:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 msgr2=0x7f4d9c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 0x7f4d9c0b4e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559015f21960 tx=0x7f4da80095c0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 msgr2=0x7f4db01a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db01a4060 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4db0074880 tx=0x7f4da000ee50 comp rx=0 tx=0).stop 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 shutdown_connections 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4db0110590 0x7f4db01ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db010dc30 0x7f4db01a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4d9c0b2930 0x7f4d9c0b4e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 --2- 172.21.15.50:0/296331627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4db0109f20 0x7f4db01a4060 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 >> 172.21.15.50:0/296331627 conn(0x7f4db00705e0 msgr2=0x7f4db010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 shutdown_connections 2024-05-17T04:47:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:20.652+0000 7f4db4fe8640 1 -- 172.21.15.50:0/296331627 wait complete. 2024-05-17T04:47:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:22 smithi105 ceph-mon[34795]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.1 MiB/s wr, 329 op/s 2024-05-17T04:47:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:22 smithi110 ceph-mon[35497]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.1 MiB/s wr, 329 op/s 2024-05-17T04:47:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:22 smithi050 ceph-mon[31739]: pgmap v770: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.1 MiB/s wr, 329 op/s 2024-05-17T04:47:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 505.0 s, 121.6 tps, lat 819.693 ms stddev 198.667 2024-05-17T04:47:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:24 smithi105 ceph-mon[34795]: pgmap v771: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.5 MiB/s wr, 331 op/s 2024-05-17T04:47:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:24 smithi110 ceph-mon[35497]: pgmap v771: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.5 MiB/s wr, 331 op/s 2024-05-17T04:47:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:24 smithi050 ceph-mon[31739]: pgmap v771: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 3.5 MiB/s wr, 331 op/s 2024-05-17T04:47:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:26 smithi105 ceph-mon[34795]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-05-17T04:47:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:26 smithi110 ceph-mon[35497]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-05-17T04:47:26.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:26 smithi050 ceph-mon[31739]: pgmap v772: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-05-17T04:47:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 510.0 s, 133.2 tps, lat 743.926 ms stddev 224.089 2024-05-17T04:47:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:28 smithi105 ceph-mon[34795]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:47:28.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:28 smithi110 ceph-mon[35497]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:47:28.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:28 smithi050 ceph-mon[31739]: pgmap v773: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:47:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:30 smithi105 ceph-mon[34795]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:47:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:30 smithi110 ceph-mon[35497]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:47:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:30 smithi050 ceph-mon[31739]: pgmap v774: 129 pgs: 129 active+clean; 1.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:47:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:32 smithi105 ceph-mon[34795]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 385 op/s 2024-05-17T04:47:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:32 smithi110 ceph-mon[35497]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 385 op/s 2024-05-17T04:47:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:32 smithi050 ceph-mon[31739]: pgmap v775: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 2.9 MiB/s wr, 385 op/s 2024-05-17T04:47:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 515.0 s, 114.6 tps, lat 869.453 ms stddev 209.823 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 -- 172.21.15.50:0/2910662889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 msgr2=0x7f5a10111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 --2- 172.21.15.50:0/2910662889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a10111b10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009ca0 tx=0x7f5a0802a8c0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 -- 172.21.15.50:0/2910662889 shutdown_connections 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 --2- 172.21.15.50:0/2910662889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a10111b10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 --2- 172.21.15.50:0/2910662889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a1010a8c0 0x7f5a1010acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 --2- 172.21.15.50:0/2910662889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a101097a0 0x7f5a10109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 -- 172.21.15.50:0/2910662889 >> 172.21.15.50:0/2910662889 conn(0x7f5a100789d0 msgr2=0x7f5a1007ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 -- 172.21.15.50:0/2910662889 shutdown_connections 2024-05-17T04:47:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.789+0000 7f5a148d1640 1 -- 172.21.15.50:0/2910662889 wait complete. 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.790+0000 7f5a148d1640 1 Processor -- start 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.790+0000 7f5a148d1640 1 -- start start 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.790+0000 7f5a148d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 0x7f5a1019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.790+0000 7f5a148d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.790+0000 7f5a148d1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a101a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a148d1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10115140 con 0x7f5a1010b210 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44698/0 (socket says 172.21.15.50:44698) 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 -- 172.21.15.50:0/3243428965 learned_addr learned my addr 172.21.15.50:0/3243428965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a101a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10114fc0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101152c0 con 0x7f5a101097a0 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 0x7f5a1019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 -- 172.21.15.50:0/3243428965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 msgr2=0x7f5a1019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 0x7f5a1019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 -- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 msgr2=0x7f5a101a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a101a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0dd9b640 1 -- 172.21.15.50:0/3243428965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101a78f0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a101a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.791+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 0x7f5a1019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e9f0 tx=0x7f5a0400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f59ff7fe640 1 -- 172.21.15.50:0/3243428965 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400cd80 con 0x7f5a1010a8c0 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101a7be0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f59ff7fe640 1 -- 172.21.15.50:0/3243428965 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a040054b0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101a80e0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.792+0000 7f59ff7fe640 1 -- 172.21.15.50:0/3243428965 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04010430 con 0x7f5a1010a8c0 2024-05-17T04:47:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.796+0000 7f5a1b783740 1 Processor -- start 2024-05-17T04:47:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.796+0000 7f59ff7fe640 1 -- 172.21.15.50:0/3243428965 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a040046c0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.796+0000 7f5a1b783740 1 -- start start 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.796+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.796+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804baacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804baacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:44708/0 (socket says 172.21.15.50:44708) 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804baacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45706/0 (socket says 172.21.15.50:45706) 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 -- 172.21.15.50:0/3477898856 learned_addr learned my addr 172.21.15.50:0/3477898856 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e804b1ccc0 con 0x55e8047de2f0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e804b0c860 con 0x55e8048071b0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e804b108a0 con 0x55e8047e64d0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 -- 172.21.15.50:0/3477898856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 msgr2=0x55e804bb1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 -- 172.21.15.50:0/3477898856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 msgr2=0x55e804baacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804baacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f59ff7fe640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 0x7f59f80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0ed9d640 1 -- 172.21.15.50:0/3477898856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e804ad4d50 con 0x55e8048071b0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.797+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f59ff7fe640 1 -- 172.21.15.50:0/3243428965 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a040e29f0 con 0x7f5a1010a8c0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 0x7f59f80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a08004ac0 tx=0x7f5a08009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caedba104efa115 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f59fcff9640 1 -- 172.21.15.50:0/3477898856 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08031d40 con 0x55e8048071b0 2024-05-17T04:47:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f59fcff9640 1 -- 172.21.15.50:0/3477898856 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a08036070 con 0x55e8048071b0 2024-05-17T04:47:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f5a0dd9b640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 0x7f59f80b4d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a6e90 tx=0x7f5a04009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.798+0000 7f59fcff9640 1 -- 172.21.15.50:0/3477898856 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080047e0 con 0x55e8048071b0 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 msgr2=0x55e804baa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a08004ac0 tx=0x7f5a08009800 comp rx=0 tx=0).stop 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 shutdown_connections 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804baacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 --2- 172.21.15.50:0/3477898856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804baa770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.799+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 >> 172.21.15.50:0/3477898856 conn(0x55e8047fb300 msgr2=0x55e804ba61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.800+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 shutdown_connections 2024-05-17T04:47:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.800+0000 7f5a1b783740 1 -- 172.21.15.50:0/3477898856 wait complete. 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.800+0000 7f5a1b783740 1 -- start start 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.800+0000 7f5a1b783740 1 Processor -- start 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804bbb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e804a48ab0 con 0x55e8047de2f0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e804ac94d0 con 0x55e8048071b0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e804bb24b0 con 0x55e8047e64d0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a1b783740 10 client.0 _ll_register_callbacks cb 0x55e804b07950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:35540/0 (socket says 172.21.15.50:35540) 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 -- 172.21.15.50:0/725761281 learned_addr learned my addr 172.21.15.50:0/725761281 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804bbb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 -- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 msgr2=0x55e804bbb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804bbb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 -- 172.21.15.50:0/725761281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 msgr2=0x55e804bb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.801+0000 7f5a0e59c640 1 -- 172.21.15.50:0/725761281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e804aca340 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009c40 tx=0x7f5a000039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd61a6103675d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f59f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00012860 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a00012a00 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a08036070 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a00012c00 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f5a1b783740 10 client.25921 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e804872790 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.802+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a08036070 con 0x55e8047e64d0 2024-05-17T04:47:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a0001a030 con 0x55e8047e64d0 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 10 client.25921 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 20 client.25921 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8047c5d80 0x55e804bc1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 4 client.25921 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e804b1ec10 con 0x55e8047c5d80 2024-05-17T04:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.803+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8047c5d80 0x55e804bc1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.804+0000 7f59f5ffb640 10 client.25921 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.804+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8047c5d80 0x55e804bc1700 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb586e4a8803621b server_cookie=1ff4ede8928b62c9 in_seq=0 out_seq=0 2024-05-17T04:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f59f5ffb640 1 -- 172.21.15.50:0/725761281 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e804b1ec10 con 0x55e8047c5d80 2024-05-17T04:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f59f5ffb640 10 client.25921 handle_command_reply: tid=0 2024-05-17T04:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 client.25921 shutdown 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 msgr2=0x55e804bb43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009c40 tx=0x7f5a000039c0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 shutdown_connections 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8048071b0 0x55e804bbb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bb43f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8047c5d80 0x55e804bc1700 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f59f5ffb640 0 client.25921 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 --2- 172.21.15.50:0/725761281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.805+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 >> 172.21.15.50:0/725761281 conn(0x55e8047fb300 msgr2=0x55e804ba9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.806+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 shutdown_connections 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.806+0000 7f5a1b783740 1 -- 172.21.15.50:0/725761281 wait complete. 2024-05-17T04:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.806+0000 7f5a1b783740 20 client.25921 trim_cache size 0 max 16384 2024-05-17T04:47:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 Processor -- start 2024-05-17T04:47:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 -- start start 2024-05-17T04:47:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047c5d80 0x55e804ba6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b4de0 con 0x55e8047de2f0 2024-05-17T04:47:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b26b0 con 0x55e8047c5d80 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.825+0000 7f5a1b783740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b0500 con 0x55e8047e64d0 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45726/0 (socket says 172.21.15.50:45726) 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 -- 172.21.15.50:0/499515903 learned_addr learned my addr 172.21.15.50:0/499515903 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047c5d80 0x55e804ba6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 -- 172.21.15.50:0/499515903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 msgr2=0x55e804bbb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bbb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 -- 172.21.15.50:0/499515903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047c5d80 msgr2=0x55e804ba6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047c5d80 0x55e804ba6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 -- 172.21.15.50:0/499515903 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8048eb620 con 0x55e8047de2f0 2024-05-17T04:47:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec6c6eb1d6385b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f59f7fff640 1 -- 172.21.15.50:0/499515903 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0002c070 con 0x55e8047de2f0 2024-05-17T04:47:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f59f7fff640 1 -- 172.21.15.50:0/499515903 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a00012860 con 0x55e8047de2f0 2024-05-17T04:47:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.826+0000 7f59f7fff640 1 -- 172.21.15.50:0/499515903 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001b720 con 0x55e8047de2f0 2024-05-17T04:47:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 -- 172.21.15.50:0/499515903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 msgr2=0x55e804bbb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00009680 comp rx=0 tx=0).stop 2024-05-17T04:47:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 -- 172.21.15.50:0/499515903 shutdown_connections 2024-05-17T04:47:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804bbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047de2f0 0x55e804bbb270 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 --2- 172.21.15.50:0/499515903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047c5d80 0x55e804ba6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.828+0000 7f5a1b783740 1 -- 172.21.15.50:0/499515903 >> 172.21.15.50:0/499515903 conn(0x55e8047fb300 msgr2=0x55e804ba5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.829+0000 7f5a1b783740 1 -- 172.21.15.50:0/499515903 shutdown_connections 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.829+0000 7f5a1b783740 1 -- 172.21.15.50:0/499515903 wait complete. 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.829+0000 7f5a1b783740 1 -- start start 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.830+0000 7f5a1b783740 1 Processor -- start 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.830+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 0x55e804bb6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.830+0000 7f5a1b783740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 0x55e804bb6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 0x55e804bb6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 0x55e804bb6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45730/0 (socket says 172.21.15.50:45730) 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a0ed9d640 1 -- 172.21.15.50:0/2731179891 learned_addr learned my addr 172.21.15.50:0/2731179891 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 0x55e804bb6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804c38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b0b70 con 0x55e8047c5d80 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b1fe0 con 0x55e8047de2f0 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101b1240 con 0x55e8047e64d0 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.831+0000 7f5a1b783740 10 client.0 _ll_register_callbacks cb 0x7f5a00004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804c38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 -- 172.21.15.50:0/2731179891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 msgr2=0x55e804c38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804c38a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 -- 172.21.15.50:0/2731179891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 msgr2=0x55e804bb6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 0x55e804bb6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e804bbbc70 con 0x55e8047de2f0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f5a0e59c640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 0x55e804bb6c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a000059f0 tx=0x7f5a0001b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e03ce86bfc938e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.832+0000 7f59f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00021040 con 0x55e8047de2f0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5a00005240 con 0x55e8047de2f0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00012860 con 0x55e8047de2f0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a0003e050 con 0x55e8047de2f0 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f5a1b783740 10 client.25946 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.833+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e804c39de0 con 0x55e8047de2f0 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a00034070 con 0x55e8047de2f0 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a0001a040 con 0x55e8047de2f0 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 10 client.25946 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 20 client.25946 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8048071b0 0x55e804c3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 4 client.25946 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5a00012860 con 0x55e8048071b0 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.834+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8048071b0 0x55e804c3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.835+0000 7f5a0ed9d640 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8048071b0 0x55e804c3c7a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5cdc91cbba702 server_cookie=60869aad4810e843 in_seq=0 out_seq=0 2024-05-17T04:47:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.835+0000 7f59f5ffb640 10 client.25946 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.876+0000 7f59f5ffb640 1 -- 172.21.15.50:0/2731179891 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f5a00012860 con 0x55e8048071b0 2024-05-17T04:47:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.876+0000 7f59f5ffb640 10 client.25946 handle_command_reply: tid=0 2024-05-17T04:47:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.876+0000 7f5a1b783740 1 client.25946 shutdown 2024-05-17T04:47:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.876+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 msgr2=0x55e804bb6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.876+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 0x55e804bb6c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5a000059f0 tx=0x7f5a0001b6f0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 shutdown_connections 2024-05-17T04:47:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e8047e64d0 0x55e804c38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8047de2f0 0x55e804bb6c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e8048071b0 0x55e804c3c7a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 --2- 172.21.15.50:0/2731179891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e8047c5d80 0x55e804bb6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 >> 172.21.15.50:0/2731179891 conn(0x55e8047fb300 msgr2=0x55e804ba8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f59f5ffb640 0 client.25946 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 shutdown_connections 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 1 -- 172.21.15.50:0/2731179891 wait complete. 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.877+0000 7f5a1b783740 20 client.25946 trim_cache size 0 max 16384 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 msgr2=0x7f59f80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 0x7f59f80b4d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a6e90 tx=0x7f5a04009720 comp rx=0 tx=0).stop 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 msgr2=0x7f5a101a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e9f0 tx=0x7f5a0400edf0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 shutdown_connections 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f59f80b2860 0x7f59f80b4d70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a1010b210 0x7f5a101a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a1010a8c0 0x7f5a101a02f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 --2- 172.21.15.50:0/3243428965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a101097a0 0x7f5a1019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 >> 172.21.15.50:0/3243428965 conn(0x7f5a100789d0 msgr2=0x7f5a1010dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 shutdown_connections 2024-05-17T04:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:33.878+0000 7f5a148d1640 1 -- 172.21.15.50:0/3243428965 wait complete. 2024-05-17T04:47:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:34 smithi105 ceph-mon[34795]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-17T04:47:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:34 smithi110 ceph-mon[35497]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-17T04:47:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:34 smithi050 ceph-mon[31739]: pgmap v776: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-17T04:47:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:36 smithi105 ceph-mon[34795]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-05-17T04:47:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:36 smithi110 ceph-mon[35497]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-05-17T04:47:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:36 smithi050 ceph-mon[31739]: pgmap v777: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-05-17T04:47:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 520.0 s, 111.8 tps, lat 897.631 ms stddev 216.313 2024-05-17T04:47:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:38 smithi105 ceph-mon[34795]: pgmap v778: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-05-17T04:47:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:38 smithi110 ceph-mon[35497]: pgmap v778: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-05-17T04:47:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:38 smithi050 ceph-mon[31739]: pgmap v778: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-05-17T04:47:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:40 smithi105 ceph-mon[34795]: pgmap v779: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:47:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:40 smithi110 ceph-mon[35497]: pgmap v779: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:47:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:40 smithi050 ceph-mon[31739]: pgmap v779: 129 pgs: 129 active+clean; 1.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.6 MiB/s wr, 366 op/s 2024-05-17T04:47:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:42 smithi105 ceph-mon[34795]: pgmap v780: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-05-17T04:47:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:42 smithi110 ceph-mon[35497]: pgmap v780: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-05-17T04:47:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:42 smithi050 ceph-mon[31739]: pgmap v780: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.4 MiB/s wr, 441 op/s 2024-05-17T04:47:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 525.0 s, 103.0 tps, lat 916.123 ms stddev 214.696 2024-05-17T04:47:43.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:43 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:47:43.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:43 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:47:43.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:43 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:47:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:44 smithi105 ceph-mon[34795]: pgmap v781: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-17T04:47:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:44 smithi110 ceph-mon[35497]: pgmap v781: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-17T04:47:44.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:44 smithi050 ceph-mon[31739]: pgmap v781: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 3.1 MiB/s wr, 405 op/s 2024-05-17T04:47:45.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:47:45.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:47:45.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:47:45.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:47:45.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:47:45.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:47:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:47:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:47:45.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:47:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:46 smithi105 ceph-mon[34795]: pgmap v782: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:47:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:46 smithi110 ceph-mon[35497]: pgmap v782: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:47:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:46 smithi050 ceph-mon[31739]: pgmap v782: 129 pgs: 129 active+clean; 1.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:47:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.024+0000 7f9e3d22d640 1 -- 172.21.15.50:0/3488624404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e38104c10 msgr2=0x7f9e3810b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.024+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/3488624404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e38104c10 0x7f9e3810b510 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009ca0 tx=0x7f9e3002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 -- 172.21.15.50:0/3488624404 shutdown_connections 2024-05-17T04:47:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/3488624404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e38104c10 0x7f9e3810b510 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/3488624404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e381042c0 0x7f9e381046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/3488624404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e381035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 -- 172.21.15.50:0/3488624404 >> 172.21.15.50:0/3488624404 conn(0x7f9e380fe9a0 msgr2=0x7f9e38100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 -- 172.21.15.50:0/3488624404 shutdown_connections 2024-05-17T04:47:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.025+0000 7f9e3d22d640 1 -- 172.21.15.50:0/3488624404 wait complete. 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 Processor -- start 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 -- start start 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e381042c0 0x7f9e381a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 0x7f9e381a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e380769c0 con 0x7f9e381042c0 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38076840 con 0x7f9e381031a0 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3d22d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38076b40 con 0x7f9e38104c10 2024-05-17T04:47:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 0x7f9e381a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36524/0 (socket says 172.21.15.50:36524) 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2627234187 learned_addr learned my addr 172.21.15.50:0/2627234187 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.026+0000 7f9e3659c640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e381042c0 0x7f9e381a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2627234187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 msgr2=0x7f9e381a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 0x7f9e381a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e381042c0 msgr2=0x7f9e381a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e381042c0 0x7f9e381a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2627234187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e381a78b0 con 0x7f9e381031a0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e3759e640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 0x7f9e381a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000e9a0 tx=0x7f9e2000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.027+0000 7f9e2ffff640 1 -- 172.21.15.50:0/2627234187 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000cd80 con 0x7f9e381031a0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.028+0000 7f9e2ffff640 1 -- 172.21.15.50:0/2627234187 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e200035f0 con 0x7f9e381031a0 2024-05-17T04:47:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.028+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e381a7ba0 con 0x7f9e381031a0 2024-05-17T04:47:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.028+0000 7f9e2ffff640 1 -- 172.21.15.50:0/2627234187 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20010690 con 0x7f9e381031a0 2024-05-17T04:47:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.028+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e381a8140 con 0x7f9e381031a0 2024-05-17T04:47:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.031+0000 7f9e441c4740 1 Processor -- start 2024-05-17T04:47:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.031+0000 7f9e441c4740 1 -- start start 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.031+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a70ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e2ffff640 1 -- 172.21.15.50:0/2627234187 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9e20010830 con 0x7f9e381031a0 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 0x564e6a715c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e441c4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564e6a680cc0 con 0x564e6a36b1b0 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e441c4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e6a670860 con 0x564e6a3422f0 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e441c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e6a6748a0 con 0x564e6a34a4d0 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e3759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e3759e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56844/0 (socket says 172.21.15.50:56844) 2024-05-17T04:47:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e3759e640 1 -- 172.21.15.50:0/4277023955 learned_addr learned my addr 172.21.15.50:0/4277023955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.032+0000 7f9e3659c640 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 0x564e6a715c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e2ffff640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 0x7f9e240b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3659c640 1 -- 172.21.15.50:0/4277023955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 msgr2=0x564e6a70ecf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3659c640 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a70ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3659c640 1 -- 172.21.15.50:0/4277023955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 msgr2=0x564e6a70e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3659c640 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3659c640 1 -- 172.21.15.50:0/4277023955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6a638d50 con 0x564e6a34a4d0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e3759e640 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.033+0000 7f9e2ffff640 1 -- 172.21.15.50:0/2627234187 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9e200e27b0 con 0x7f9e381031a0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.034+0000 7f9e3659c640 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 0x564e6a715c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009c20 tx=0x7f9e28011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d266bae76363e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.034+0000 7f9e3759e640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 0x7f9e240b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.034+0000 7f9e2d7fa640 1 -- 172.21.15.50:0/4277023955 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2801e070 con 0x564e6a34a4d0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.034+0000 7f9e2d7fa640 1 -- 172.21.15.50:0/4277023955 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e28009dd0 con 0x564e6a34a4d0 2024-05-17T04:47:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.034+0000 7f9e2d7fa640 1 -- 172.21.15.50:0/4277023955 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e280186a0 con 0x564e6a34a4d0 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- 172.21.15.50:0/4277023955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 msgr2=0x564e6a715c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 0x564e6a715c10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009c20 tx=0x7f9e28011810 comp rx=0 tx=0).stop 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e3759e640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 0x7f9e240b4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e30031880 tx=0x7f9e300095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- 172.21.15.50:0/4277023955 shutdown_connections 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a34a4d0 0x564e6a715c10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a70ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 --2- 172.21.15.50:0/4277023955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a36b1b0 0x564e6a70e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- 172.21.15.50:0/4277023955 >> 172.21.15.50:0/4277023955 conn(0x564e6a35f300 msgr2=0x564e6a70a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- 172.21.15.50:0/4277023955 shutdown_connections 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- 172.21.15.50:0/4277023955 wait complete. 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.035+0000 7f9e441c4740 1 -- start start 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 Processor -- start 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a718140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 0x564e6a71f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564e6a5acab0 con 0x564e6a34a4d0 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e6a62d4d0 con 0x564e6a3422f0 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564e6a716730 con 0x564e6a36b1b0 2024-05-17T04:47:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e441c4740 10 client.0 _ll_register_callbacks cb 0x564e6a66b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e3659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a718140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e3659c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56850/0 (socket says 172.21.15.50:56850) 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e3659c640 1 -- 172.21.15.50:0/3288289266 learned_addr learned my addr 172.21.15.50:0/3288289266 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e36d9d640 1 -- 172.21.15.50:0/3288289266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 msgr2=0x564e6a71f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.036+0000 7f9e3759e640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 0x564e6a71f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 0x564e6a71f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e36d9d640 1 -- 172.21.15.50:0/3288289266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 msgr2=0x564e6a718680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e36d9d640 1 -- 172.21.15.50:0/3288289266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6a62e340 con 0x564e6a3422f0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e3659c640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e3759e640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 0x564e6a71f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a718140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564e6a70fb70 tx=0x7f9e200a9480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6866c25cc32b3792 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200b2200 con 0x564e6a3422f0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9e200a5cd0 con 0x564e6a3422f0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e28009dd0 con 0x564e6a3422f0 2024-05-17T04:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e200a9550 con 0x564e6a3422f0 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e441c4740 10 client.25955 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.037+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564e6a3d6790 con 0x564e6a3422f0 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e28009dd0 con 0x564e6a3422f0 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9e200a8330 con 0x564e6a3422f0 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 10 client.25955 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 20 client.25955 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a329d80 0x564e6a7258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 4 client.25955 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.038+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e6a682c10 con 0x564e6a329d80 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.039+0000 7f9e3659c640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a329d80 0x564e6a7258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.039+0000 7f9e1e7fc640 10 client.25955 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.040+0000 7f9e3659c640 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a329d80 0x564e6a7258c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6b91cde2b85431 server_cookie=ee6aff8a2bb54295 in_seq=0 out_seq=0 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3288289266 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e6a682c10 con 0x564e6a329d80 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e1e7fc640 10 client.25955 handle_command_reply: tid=0 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 client.25955 shutdown 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 msgr2=0x564e6a718140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a718140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x564e6a70fb70 tx=0x7f9e200a9480 comp rx=0 tx=0).stop 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 shutdown_connections 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a36b1b0 0x564e6a71f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a329d80 0x564e6a7258c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a34a4d0 0x564e6a718680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3288289266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a3422f0 0x564e6a718140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e1e7fc640 0 client.25955 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 >> 172.21.15.50:0/3288289266 conn(0x564e6a35f300 msgr2=0x564e6a7091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 shutdown_connections 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.041+0000 7f9e441c4740 1 -- 172.21.15.50:0/3288289266 wait complete. 2024-05-17T04:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.042+0000 7f9e441c4740 20 client.25955 trim_cache size 0 max 16384 2024-05-17T04:47:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.068+0000 7f9e441c4740 1 Processor -- start 2024-05-17T04:47:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.068+0000 7f9e441c4740 1 -- start start 2024-05-17T04:47:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.068+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 0x564e6a70cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 0x564e6a70d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e441c4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b4db0 con 0x564e6a329d80 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e441c4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b2680 con 0x564e6a35f300 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36554/0 (socket says 172.21.15.50:36554) 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2612873775 learned_addr learned my addr 172.21.15.50:0/2612873775 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b04d0 con 0x564e6a3422f0 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e3659c640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 0x564e6a70cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e3759e640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 0x564e6a70d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2612873775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 msgr2=0x564e6a70d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 0x564e6a70d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2612873775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 msgr2=0x564e6a70cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 0x564e6a70cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e36d9d640 1 -- 172.21.15.50:0/2612873775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6a44f620 con 0x564e6a35f300 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e3759e640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 0x564e6a70d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.069+0000 7f9e3659c640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 0x564e6a70cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.070+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e200a96f0 tx=0x7f9e20003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae2d1fb5706cf9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.070+0000 7f9e2cff9640 1 -- 172.21.15.50:0/2612873775 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200b2200 con 0x564e6a35f300 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.070+0000 7f9e2cff9640 1 -- 172.21.15.50:0/2612873775 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e200aa140 con 0x564e6a35f300 2024-05-17T04:47:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.070+0000 7f9e2cff9640 1 -- 172.21.15.50:0/2612873775 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e200a8000 con 0x564e6a35f300 2024-05-17T04:47:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 msgr2=0x564e6a70a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e200a96f0 tx=0x7f9e20003d70 comp rx=0 tx=0).stop 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 shutdown_connections 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a3422f0 0x564e6a70d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a329d80 0x564e6a70cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 --2- 172.21.15.50:0/2612873775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a70a200 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 >> 172.21.15.50:0/2612873775 conn(0x564e6a725c30 msgr2=0x564e6a709af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 shutdown_connections 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.071+0000 7f9e441c4740 1 -- 172.21.15.50:0/2612873775 wait complete. 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 -- start start 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 Processor -- start 2024-05-17T04:47:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a3422f0 0x564e6a71c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a71d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b0b40 con 0x564e6a3422f0 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b1fb0 con 0x564e6a35f300 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e381b1210 con 0x564e6a329d80 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.072+0000 7f9e441c4740 10 client.0 _ll_register_callbacks cb 0x7f9e2000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a71d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e36d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:53674/0 (socket says 172.21.15.50:53674) 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e36d9d640 1 -- 172.21.15.50:0/3417519377 learned_addr learned my addr 172.21.15.50:0/3417519377 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:47:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3659c640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a3422f0 0x564e6a71c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 -- 172.21.15.50:0/3417519377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 msgr2=0x564e6a71bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 -- 172.21.15.50:0/3417519377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a3422f0 msgr2=0x564e6a71c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:47:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a3422f0 0x564e6a71c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 -- 172.21.15.50:0/3417519377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6a71d970 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e3759e640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a71d380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005e30 tx=0x7f9e30042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b222a0d43651bd31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.073+0000 7f9e1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30059060 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e200aa140 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9e30047830 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e30047a30 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e441c4740 10 client.25961 fetch_fsmap learned FSMap version 39 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564e6a71eca0 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.074+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e200aa140 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9e30046420 con 0x564e6a35f300 2024-05-17T04:47:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 10 client.25961 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 20 client.25961 populate_metadata read hostname 'smithi050' 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a34a4d0 0x564e6a7a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 4 client.25961 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e6a67c3f0 con 0x564e6a34a4d0 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a34a4d0 0x564e6a7a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.075+0000 7f9e1e7fc640 10 client.25961 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.076+0000 7f9e36d9d640 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a34a4d0 0x564e6a7a1180 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19eb1e854e957f54 server_cookie=f021e936442815b5 in_seq=0 out_seq=0 2024-05-17T04:47:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e1e7fc640 1 -- 172.21.15.50:0/3417519377 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564e6a67c3f0 con 0x564e6a34a4d0 2024-05-17T04:47:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e1e7fc640 10 client.25961 handle_command_reply: tid=0 2024-05-17T04:47:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 client.25961 shutdown 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 msgr2=0x564e6a71d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a71d380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005e30 tx=0x7f9e30042000 comp rx=0 tx=0).stop 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 shutdown_connections 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e6a35f300 0x564e6a71d380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x564e6a34a4d0 0x564e6a7a1180 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564e6a3422f0 0x564e6a71c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e1e7fc640 0 client.25961 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 --2- 172.21.15.50:0/3417519377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564e6a329d80 0x564e6a71bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 >> 172.21.15.50:0/3417519377 conn(0x564e6a725c30 msgr2=0x564e6a7087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 shutdown_connections 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 1 -- 172.21.15.50:0/3417519377 wait complete. 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.110+0000 7f9e441c4740 20 client.25961 trim_cache size 0 max 16384 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:47:47.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:47:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 msgr2=0x7f9e240b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 0x7f9e240b4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e30031880 tx=0x7f9e300095c0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 msgr2=0x7f9e3819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000e9a0 tx=0x7f9e2000ee60 comp rx=0 tx=0).stop 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 shutdown_connections 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e38104c10 0x7f9e381a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9e240b2850 0x7f9e240b4d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e381042c0 0x7f9e381a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 --2- 172.21.15.50:0/2627234187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e381031a0 0x7f9e3819fd80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 >> 172.21.15.50:0/2627234187 conn(0x7f9e380fe9a0 msgr2=0x7f9e381075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 shutdown_connections 2024-05-17T04:47:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:47:47.112+0000 7f9e3d22d640 1 -- 172.21.15.50:0/2627234187 wait complete. 2024-05-17T04:47:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 530.0 s, 131.4 tps, lat 808.785 ms stddev 330.694 2024-05-17T04:47:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:48 smithi105 ceph-mon[34795]: pgmap v783: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:47:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:48 smithi110 ceph-mon[35497]: pgmap v783: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:47:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:48 smithi050 ceph-mon[31739]: pgmap v783: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:47:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:50 smithi105 ceph-mon[34795]: pgmap v784: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:47:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:50 smithi110 ceph-mon[35497]: pgmap v784: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:47:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:50 smithi050 ceph-mon[31739]: pgmap v784: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:47:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:52 smithi105 ceph-mon[34795]: pgmap v785: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-05-17T04:47:52.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:52 smithi110 ceph-mon[35497]: pgmap v785: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-05-17T04:47:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:52 smithi050 ceph-mon[31739]: pgmap v785: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-05-17T04:47:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 535.0 s, 126.6 tps, lat 795.062 ms stddev 192.867 2024-05-17T04:47:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:54 smithi105 ceph-mon[34795]: pgmap v786: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-17T04:47:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:54 smithi110 ceph-mon[35497]: pgmap v786: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-17T04:47:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:54 smithi050 ceph-mon[31739]: pgmap v786: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 2.8 MiB/s wr, 397 op/s 2024-05-17T04:47:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:56 smithi105 ceph-mon[34795]: pgmap v787: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.8 MiB/s wr, 392 op/s 2024-05-17T04:47:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:56 smithi110 ceph-mon[35497]: pgmap v787: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.8 MiB/s wr, 392 op/s 2024-05-17T04:47:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:56 smithi050 ceph-mon[31739]: pgmap v787: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 2.8 MiB/s wr, 392 op/s 2024-05-17T04:47:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 540.0 s, 113.4 tps, lat 861.006 ms stddev 201.754 2024-05-17T04:47:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:47:58 smithi105 ceph-mon[34795]: pgmap v788: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:47:58.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:47:58 smithi110 ceph-mon[35497]: pgmap v788: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:47:58.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:47:58 smithi050 ceph-mon[31739]: pgmap v788: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.8 MiB/s wr, 396 op/s 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 -- 172.21.15.50:0/3003649417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101170 msgr2=0x7f8c1c101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 --2- 172.21.15.50:0/3003649417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101170 0x7f8c1c101580 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009930 tx=0x7f8c0c02a420 comp rx=0 tx=0).stop 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 -- 172.21.15.50:0/3003649417 shutdown_connections 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 --2- 172.21.15.50:0/3003649417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c10b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 --2- 172.21.15.50:0/3003649417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101170 0x7f8c1c101580 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 --2- 172.21.15.50:0/3003649417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 0x7f8c1c104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.260+0000 7f8c23f31640 1 -- 172.21.15.50:0/3003649417 >> 172.21.15.50:0/3003649417 conn(0x7f8c1c0fd010 msgr2=0x7f8c1c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.261+0000 7f8c23f31640 1 -- 172.21.15.50:0/3003649417 shutdown_connections 2024-05-17T04:48:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.261+0000 7f8c23f31640 1 -- 172.21.15.50:0/3003649417 wait complete. 2024-05-17T04:48:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.261+0000 7f8c23f31640 1 Processor -- start 2024-05-17T04:48:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.261+0000 7f8c23f31640 1 -- start start 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c23f31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 0x7f8c1c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c23f31640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c23f31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 0x7f8c1c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 0x7f8c1c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 0x7f8c1c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45682/0 (socket says 172.21.15.50:45682) 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c21ccd640 1 -- 172.21.15.50:0/1269208683 learned_addr learned my addr 172.21.15.50:0/1269208683 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c224ce640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 0x7f8c1c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.262+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c10ebc0 con 0x7f8c1c101ac0 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c10ea40 con 0x7f8c1c101170 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c10ed40 con 0x7f8c1c104730 2024-05-17T04:48:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 -- 172.21.15.50:0/1269208683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 msgr2=0x7f8c1c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 0x7f8c1c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 -- 172.21.15.50:0/1269208683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 msgr2=0x7f8c1c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 0x7f8c1c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c1a1340 con 0x7f8c1c101ac0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.263+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c199d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a940 tx=0x7f8c0c005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.264+0000 7f8c12ffd640 1 -- 172.21.15.50:0/1269208683 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c035030 con 0x7f8c1c101ac0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.264+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c102280 con 0x7f8c1c101ac0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.264+0000 7f8c12ffd640 1 -- 172.21.15.50:0/1269208683 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c0c041070 con 0x7f8c1c101ac0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.264+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c102790 con 0x7f8c1c101ac0 2024-05-17T04:48:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.264+0000 7f8c12ffd640 1 -- 172.21.15.50:0/1269208683 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c031b90 con 0x7f8c1c101ac0 2024-05-17T04:48:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.267+0000 7f8c2adee740 1 Processor -- start 2024-05-17T04:48:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 -- start start 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c12ffd640 1 -- 172.21.15.50:0/1269208683 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8c0c046470 con 0x7f8c1c101ac0 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 0x55f18c8c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f18c832cc0 con 0x55f18c4f42f0 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f18c822860 con 0x55f18c4fc4d0 2024-05-17T04:48:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c2adee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f18c8268a0 con 0x55f18c51d1b0 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c214cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c224ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 0x55f18c8c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c214cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45684/0 (socket says 172.21.15.50:45684) 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c224ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 0x55f18c8c0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44478/0 (socket says 172.21.15.50:44478) 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c214cc640 1 -- 172.21.15.50:0/3615874077 learned_addr learned my addr 172.21.15.50:0/3615874077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.268+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 -- 172.21.15.50:0/3615874077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 msgr2=0x55f18c8c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 0x55f18c8c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 -- 172.21.15.50:0/3615874077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 msgr2=0x55f18c8c0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8c0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 -- 172.21.15.50:0/3615874077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c7ead50 con 0x55f18c4fc4d0 2024-05-17T04:48:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c214cc640 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c10afa0 tx=0x7f8c0c10f1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1632da417062f3db server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c12ffd640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 0x7f8c080b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.269+0000 7f8c07fff640 1 -- 172.21.15.50:0/3615874077 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0ca800 con 0x55f18c4fc4d0 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.270+0000 7f8c224ce640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 0x7f8c080b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.270+0000 7f8c12ffd640 1 -- 172.21.15.50:0/1269208683 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8c0c106e90 con 0x7f8c1c101ac0 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.270+0000 7f8c07fff640 1 -- 172.21.15.50:0/3615874077 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c0c114070 con 0x55f18c4fc4d0 2024-05-17T04:48:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.270+0000 7f8c07fff640 1 -- 172.21.15.50:0/3615874077 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0cdb00 con 0x55f18c4fc4d0 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c224ce640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 0x7f8c080b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f18c8c1960 tx=0x7f8c14009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 -- 172.21.15.50:0/3615874077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 msgr2=0x55f18c8c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c10afa0 tx=0x7f8c0c10f1a0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 -- 172.21.15.50:0/3615874077 shutdown_connections 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8c7c10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8c0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 --2- 172.21.15.50:0/3615874077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c51d1b0 0x55f18c8c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 -- 172.21.15.50:0/3615874077 >> 172.21.15.50:0/3615874077 conn(0x55f18c511300 msgr2=0x55f18c8bc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 -- 172.21.15.50:0/3615874077 shutdown_connections 2024-05-17T04:48:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.271+0000 7f8c2adee740 1 -- 172.21.15.50:0/3615874077 wait complete. 2024-05-17T04:48:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.272+0000 7f8c2adee740 1 -- start start 2024-05-17T04:48:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.272+0000 7f8c2adee740 1 Processor -- start 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44480/0 (socket says 172.21.15.50:44480) 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c214cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c214cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45694/0 (socket says 172.21.15.50:45694) 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c21ccd640 1 -- 172.21.15.50:0/1934860262 learned_addr learned my addr 172.21.15.50:0/1934860262 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c51d1b0 0x55f18c8d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.273+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f18c75eab0 con 0x55f18c51d1b0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f18c7df4d0 con 0x55f18c4fc4d0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f18c8c8450 con 0x55f18c4f42f0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c224ce640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c51d1b0 0x55f18c8d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c2adee740 10 client.0 _ll_register_callbacks cb 0x55f18c81d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c21ccd640 1 -- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 msgr2=0x55f18c8ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c21ccd640 1 -- 172.21.15.50:0/1934860262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c51d1b0 msgr2=0x55f18c8d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c51d1b0 0x55f18c8d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.274+0000 7f8c21ccd640 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c7e0340 con 0x55f18c4f42f0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.275+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c114070 con 0x55f18c4f42f0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.275+0000 7f8c057fa640 10 client.25954 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.275+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c18012840 tx=0x7f8c18012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f501ebf6ce5167da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.275+0000 7f8c057fa640 1 -- 172.21.15.50:0/1934860262 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800f920 con 0x55f18c4f42f0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.275+0000 7f8c057fa640 1 -- 172.21.15.50:0/1934860262 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8c18018070 con 0x55f18c4f42f0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c057fa640 1 -- 172.21.15.50:0/1934860262 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c1800f920 con 0x55f18c4f42f0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 10 client.25954 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55f18c588790 con 0x55f18c4f42f0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c057fa640 1 -- 172.21.15.50:0/1934860262 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8c1800fac0 con 0x55f18c4f42f0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 10 client.25954 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 20 client.25954 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c4dbd80 0x55f18c8d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 4 client.25954 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18c834c10 con 0x55f18c4dbd80 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.276+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c4dbd80 0x55f18c8d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.277+0000 7f8c214cc640 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c4dbd80 0x55f18c8d73c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83cc2d5f9441df6 server_cookie=44c9078840f81277 in_seq=0 out_seq=0 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.277+0000 7f8c057fa640 10 client.25954 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c057fa640 1 -- 172.21.15.50:0/1934860262 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f18c834c10 con 0x55f18c4dbd80 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c057fa640 10 client.25954 handle_command_reply: tid=0 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 client.25954 shutdown 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 msgr2=0x55f18c8c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c18012840 tx=0x7f8c18012d00 comp rx=0 tx=0).stop 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 shutdown_connections 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c4dbd80 0x55f18c8d73c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c057fa640 0 client.25954 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c51d1b0 0x55f18c8d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 --2- 172.21.15.50:0/1934860262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4f42f0 0x55f18c8c9d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.278+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 >> 172.21.15.50:0/1934860262 conn(0x55f18c511300 msgr2=0x55f18c8bb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.279+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 shutdown_connections 2024-05-17T04:48:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.279+0000 7f8c2adee740 1 -- 172.21.15.50:0/1934860262 wait complete. 2024-05-17T04:48:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.279+0000 7f8c2adee740 20 client.25954 trim_cache size 0 max 16384 2024-05-17T04:48:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.300+0000 7f8c2adee740 1 Processor -- start 2024-05-17T04:48:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.301+0000 7f8c2adee740 1 -- start start 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.301+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.301+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.301+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c2adee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1ac800 con 0x55f18c4f42f0 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c21ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45710/0 (socket says 172.21.15.50:45710) 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c21ccd640 1 -- 172.21.15.50:0/294730261 learned_addr learned my addr 172.21.15.50:0/294730261 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1aa0d0 con 0x55f18c4dbd80 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1a7ec0 con 0x55f18c4fc4d0 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c214cc640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c224ce640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.302+0000 7f8c21ccd640 1 -- 172.21.15.50:0/294730261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4fc4d0 msgr2=0x55f18c8d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c21ccd640 1 -- 172.21.15.50:0/294730261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 msgr2=0x55f18c8d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c21ccd640 1 -- 172.21.15.50:0/294730261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c601620 con 0x55f18c4dbd80 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c214cc640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c1801bb00 tx=0x7f8c18014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2805ecbf99dd1a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.303+0000 7f8c077fe640 1 -- 172.21.15.50:0/294730261 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18000e60 con 0x55f18c4dbd80 2024-05-17T04:48:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.304+0000 7f8c077fe640 1 -- 172.21.15.50:0/294730261 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c0c114070 con 0x55f18c4dbd80 2024-05-17T04:48:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.304+0000 7f8c077fe640 1 -- 172.21.15.50:0/294730261 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18000e60 con 0x55f18c4dbd80 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 msgr2=0x55f18c8baef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c1801bb00 tx=0x7f8c18014000 comp rx=0 tx=0).stop 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 shutdown_connections 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 --2- 172.21.15.50:0/294730261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4dbd80 0x55f18c8baef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.305+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 >> 172.21.15.50:0/294730261 conn(0x55f18c511300 msgr2=0x55f18c8ba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.306+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 shutdown_connections 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.306+0000 7f8c2adee740 1 -- 172.21.15.50:0/294730261 wait complete. 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.306+0000 7f8c2adee740 1 -- start start 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.306+0000 7f8c2adee740 1 Processor -- start 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4dbd80 0x55f18c8cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1a8590 con 0x55f18c4f42f0 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1a9a00 con 0x55f18c4fc4d0 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1a8c60 con 0x55f18c4dbd80 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c224ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c2adee740 10 client.0 _ll_register_callbacks cb 0x7f8c18009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c224ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45726/0 (socket says 172.21.15.50:45726) 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.307+0000 7f8c224ce640 1 -- 172.21.15.50:0/670703373 learned_addr learned my addr 172.21.15.50:0/670703373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4dbd80 0x55f18c8cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c214cc640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 -- 172.21.15.50:0/670703373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4dbd80 msgr2=0x55f18c8cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4dbd80 0x55f18c8cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 -- 172.21.15.50:0/670703373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 msgr2=0x55f18c8cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 -- 172.21.15.50:0/670703373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c8d0690 con 0x55f18c4fc4d0 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c224ce640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004260 tx=0x7f8c14004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f74426cb5d25ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.308+0000 7f8c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1402d060 con 0x55f18c4fc4d0 2024-05-17T04:48:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8c1400f6f0 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0c114070 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c14034050 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c2adee740 10 client.25979 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.309+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f18c8d19f0 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c1400fd80 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8c14027070 con 0x55f18c4fc4d0 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 10 client.25979 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 20 client.25979 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c51d1b0 0x55f18c94a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 4 client.25979 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:48:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.310+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8c0c114070 con 0x55f18c51d1b0 2024-05-17T04:48:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.311+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c51d1b0 0x55f18c94a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.311+0000 7f8c057fa640 10 client.25979 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:00.311+0000 7f8c21ccd640 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c51d1b0 0x55f18c94a390 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f991c4423b9494db server_cookie=371795ebfe89d607 in_seq=0 out_seq=0 2024-05-17T04:48:00.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:00 smithi105 ceph-mon[34795]: pgmap v789: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 2.6 MiB/s wr, 377 op/s 2024-05-17T04:48:00.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:00 smithi110 ceph-mon[35497]: pgmap v789: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 2.6 MiB/s wr, 377 op/s 2024-05-17T04:48:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:00 smithi050 ceph-mon[31739]: pgmap v789: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 2.6 MiB/s wr, 377 op/s 2024-05-17T04:48:02.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:02 smithi105 ceph-mon[34795]: pgmap v790: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-05-17T04:48:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:02 smithi110 ceph-mon[35497]: pgmap v790: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-05-17T04:48:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:02 smithi050 ceph-mon[31739]: pgmap v790: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.0 MiB/s wr, 404 op/s 2024-05-17T04:48:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 545.0 s, 118.6 tps, lat 842.859 ms stddev 223.747 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.727+0000 7f8c057fa640 1 -- 172.21.15.50:0/670703373 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f8c0c114070 con 0x55f18c51d1b0 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.727+0000 7f8c057fa640 10 client.25979 handle_command_reply: tid=0 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.727+0000 7f8c2adee740 1 client.25979 shutdown 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 msgr2=0x55f18c8d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004260 tx=0x7f8c14004ab0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 shutdown_connections 2024-05-17T04:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f18c4fc4d0 0x55f18c8d00a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f18c51d1b0 0x55f18c94a390 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f18c4f42f0 0x55f18c8cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 --2- 172.21.15.50:0/670703373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c4dbd80 0x55f18c8cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 >> 172.21.15.50:0/670703373 conn(0x55f18c511300 msgr2=0x55f18c8beb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c057fa640 0 client.25979 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 shutdown_connections 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 1 -- 172.21.15.50:0/670703373 wait complete. 2024-05-17T04:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.728+0000 7f8c2adee740 20 client.25979 trim_cache size 0 max 16384 2024-05-17T04:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 msgr2=0x7f8c080b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 0x7f8c080b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55f18c8c1960 tx=0x7f8c14009500 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 msgr2=0x7f8c1c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c199d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c02a940 tx=0x7f8c0c005d70 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 shutdown_connections 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c1c104730 0x7f8c1c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8c080b2860 0x7f8c080b4d70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8c1c101ac0 0x7f8c1c199d40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 --2- 172.21.15.50:0/1269208683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c101170 0x7f8c1c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 >> 172.21.15.50:0/1269208683 conn(0x7f8c1c0fd010 msgr2=0x7f8c1c107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 shutdown_connections 2024-05-17T04:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:03.729+0000 7f8c23f31640 1 -- 172.21.15.50:0/1269208683 wait complete. 2024-05-17T04:48:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:04 smithi105 ceph-mon[34795]: pgmap v791: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.7 MiB/s wr, 412 op/s 2024-05-17T04:48:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:04 smithi110 ceph-mon[35497]: pgmap v791: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.7 MiB/s wr, 412 op/s 2024-05-17T04:48:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:04 smithi050 ceph-mon[31739]: pgmap v791: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.7 MiB/s wr, 412 op/s 2024-05-17T04:48:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:06 smithi105 ceph-mon[34795]: pgmap v792: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 349 op/s 2024-05-17T04:48:06.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:06 smithi110 ceph-mon[35497]: pgmap v792: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 349 op/s 2024-05-17T04:48:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:06 smithi050 ceph-mon[31739]: pgmap v792: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.3 MiB/s wr, 349 op/s 2024-05-17T04:48:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 550.0 s, 121.4 tps, lat 848.542 ms stddev 215.505 2024-05-17T04:48:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:08 smithi105 ceph-mon[34795]: pgmap v793: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:48:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:08 smithi110 ceph-mon[35497]: pgmap v793: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:48:08.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:08 smithi050 ceph-mon[31739]: pgmap v793: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.5 MiB/s wr, 372 op/s 2024-05-17T04:48:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:10 smithi105 ceph-mon[34795]: pgmap v794: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:48:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:10 smithi110 ceph-mon[35497]: pgmap v794: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:48:10.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:10 smithi050 ceph-mon[31739]: pgmap v794: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 2.4 MiB/s wr, 350 op/s 2024-05-17T04:48:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:12 smithi105 ceph-mon[34795]: pgmap v795: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-05-17T04:48:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:12 smithi110 ceph-mon[35497]: pgmap v795: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-05-17T04:48:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:12 smithi050 ceph-mon[31739]: pgmap v795: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-05-17T04:48:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 555.0 s, 136.0 tps, lat 694.018 ms stddev 220.931 2024-05-17T04:48:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:14 smithi105 ceph-mon[34795]: pgmap v796: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-05-17T04:48:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:14 smithi110 ceph-mon[35497]: pgmap v796: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-05-17T04:48:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:14 smithi050 ceph-mon[31739]: pgmap v796: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.5 MiB/s wr, 388 op/s 2024-05-17T04:48:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:16 smithi105 ceph-mon[34795]: pgmap v797: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-05-17T04:48:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:16 smithi110 ceph-mon[35497]: pgmap v797: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-05-17T04:48:16.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:16 smithi050 ceph-mon[31739]: pgmap v797: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 3.0 MiB/s wr, 388 op/s 2024-05-17T04:48:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 -- 172.21.15.50:0/2726414332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 msgr2=0x7f76b81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/2726414332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b81035c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009900 tx=0x7f76ac02a410 comp rx=0 tx=0).stop 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 -- 172.21.15.50:0/2726414332 shutdown_connections 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/2726414332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8103b00 0x7f76b810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/2726414332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b81035c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/2726414332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8102090 0x7f76b81024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 -- 172.21.15.50:0/2726414332 >> 172.21.15.50:0/2726414332 conn(0x7f76b80fd8b0 msgr2=0x7f76b80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 -- 172.21.15.50:0/2726414332 shutdown_connections 2024-05-17T04:48:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.866+0000 7f76bfdd2640 1 -- 172.21.15.50:0/2726414332 wait complete. 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 Processor -- start 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 -- start start 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 0x7f76b8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8103b00 0x7f76b81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810da70 con 0x7f76b81031b0 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810d8f0 con 0x7f76b8102090 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bfdd2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810dbf0 con 0x7f76b8103b00 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bd36d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.867+0000 7f76bd36d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53298/0 (socket says 172.21.15.50:53298) 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 -- 172.21.15.50:0/3079355094 learned_addr learned my addr 172.21.15.50:0/3079355094 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 0x7f76b8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76be36f640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8103b00 0x7f76b81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 -- 172.21.15.50:0/3079355094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8103b00 msgr2=0x7f76b81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8103b00 0x7f76b81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 -- 172.21.15.50:0/3079355094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 msgr2=0x7f76b8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 0x7f76b8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 -- 172.21.15.50:0/3079355094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81a1350 con 0x7f76b81031b0 2024-05-17T04:48:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 0x7f76b8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76bd36d640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009230 tx=0x7f76ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.868+0000 7f76b6ffd640 1 -- 172.21.15.50:0/3079355094 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac03f030 con 0x7f76b81031b0 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.869+0000 7f76b6ffd640 1 -- 172.21.15.50:0/3079355094 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76ac004440 con 0x7f76b81031b0 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.869+0000 7f76b6ffd640 1 -- 172.21.15.50:0/3079355094 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac035070 con 0x7f76b81031b0 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.869+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81a15e0 con 0x7f76b81031b0 2024-05-17T04:48:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.869+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a1a80 con 0x7f76b81031b0 2024-05-17T04:48:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.872+0000 7f76c6d66740 1 Processor -- start 2024-05-17T04:48:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.872+0000 7f76c6d66740 1 -- start start 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cbec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5c29cc0 con 0x55c9a59141b0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5c19860 con 0x55c9a58eb2f0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76c6d66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5c1d8a0 con 0x55c9a58f34d0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53306/0 (socket says 172.21.15.50:53306) 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76bdb6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 -- 172.21.15.50:0/434825822 learned_addr learned my addr 172.21.15.50:0/434825822 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76b6ffd640 1 -- 172.21.15.50:0/3079355094 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f76ac042030 con 0x7f76b81031b0 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 -- 172.21.15.50:0/434825822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 msgr2=0x55c9a5cbec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cbec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 -- 172.21.15.50:0/434825822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 msgr2=0x55c9a5cb7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 -- 172.21.15.50:0/434825822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a5be1d50 con 0x55c9a59141b0 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.873+0000 7f76be36f640 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76b0011440 tx=0x7f76b0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeb9ee6786fb8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.874+0000 7f769ffff640 1 -- 172.21.15.50:0/434825822 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0003c30 con 0x55c9a59141b0 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.874+0000 7f769ffff640 1 -- 172.21.15.50:0/434825822 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76b0003dd0 con 0x55c9a59141b0 2024-05-17T04:48:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.874+0000 7f769ffff640 1 -- 172.21.15.50:0/434825822 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b000f8e0 con 0x55c9a59141b0 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.874+0000 7f76b6ffd640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 0x7f76a00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 0x7f76a00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76b6ffd640 1 -- 172.21.15.50:0/3079355094 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f76ac1029b0 con 0x7f76b81031b0 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 -- 172.21.15.50:0/434825822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 msgr2=0x55c9a5cb7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76b0011440 tx=0x7f76b0011900 comp rx=0 tx=0).stop 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 0x7f76a00b4d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009fd0 tx=0x7f76a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 -- 172.21.15.50:0/434825822 shutdown_connections 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cbec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 --2- 172.21.15.50:0/434825822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cb7770 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 -- 172.21.15.50:0/434825822 >> 172.21.15.50:0/434825822 conn(0x55c9a5908300 msgr2=0x55c9a5cb31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 -- 172.21.15.50:0/434825822 shutdown_connections 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.875+0000 7f76c6d66740 1 -- 172.21.15.50:0/434825822 wait complete. 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 -- start start 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 Processor -- start 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a59141b0 0x55c9a5cc86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5b55ab0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5bd64d0 con 0x55c9a58f34d0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.876+0000 7f76c6d66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c9a5cbf690 con 0x55c9a59141b0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53316/0 (socket says 172.21.15.50:53316) 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76c6d66740 10 client.0 _ll_register_callbacks cb 0x55c9a5c14950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 -- 172.21.15.50:0/598244816 learned_addr learned my addr 172.21.15.50:0/598244816 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bd36d640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76be36f640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a59141b0 0x55c9a5cc86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 -- 172.21.15.50:0/598244816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a59141b0 msgr2=0x55c9a5cc86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a59141b0 0x55c9a5cc86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 -- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 msgr2=0x55c9a5cc1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 -- 172.21.15.50:0/598244816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a5bd7340 con 0x55c9a58eb2f0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009d50 tx=0x7f76a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165993e0cc913a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.877+0000 7f769d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004050 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f76a80041f0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0003dd0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76a80043f0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f76c6d66740 10 client.16233 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9a597f790 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.878+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76b0003dd0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f76a8027020 con 0x55c9a58eb2f0 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 10 client.16233 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 20 client.16233 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58d2d80 0x55c9a5ccea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 4 client.16233 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:48:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9a5c2bc10 con 0x55c9a58d2d80 2024-05-17T04:48:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f76bd36d640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58d2d80 0x55c9a5ccea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.879+0000 7f769d7fa640 10 client.16233 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.880+0000 7f76bd36d640 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58d2d80 0x55c9a5ccea90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b64f4b674902d2 server_cookie=81c510705a626107 in_seq=0 out_seq=0 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f769d7fa640 1 -- 172.21.15.50:0/598244816 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9a5c2bc10 con 0x55c9a58d2d80 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f769d7fa640 10 client.16233 handle_command_reply: tid=0 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 client.16233 shutdown 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 msgr2=0x55c9a5cc1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009d50 tx=0x7f76a8004730 comp rx=0 tx=0).stop 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 shutdown_connections 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a59141b0 0x55c9a5cc86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58d2d80 0x55c9a5ccea90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 --2- 172.21.15.50:0/598244816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc1210 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 >> 172.21.15.50:0/598244816 conn(0x55c9a5908300 msgr2=0x55c9a5cb21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f769d7fa640 0 client.16233 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 shutdown_connections 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.881+0000 7f76c6d66740 1 -- 172.21.15.50:0/598244816 wait complete. 2024-05-17T04:48:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.882+0000 7f76c6d66740 20 client.16233 trim_cache size 0 max 16384 2024-05-17T04:48:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 Processor -- start 2024-05-17T04:48:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 -- start start 2024-05-17T04:48:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81ae880 con 0x55c9a59141b0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81ac150 con 0x55c9a58f34d0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.901+0000 7f76c6d66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81a9fa0 con 0x55c9a58eb2f0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53324/0 (socket says 172.21.15.50:53324) 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76be36f640 1 -- 172.21.15.50:0/310827801 learned_addr learned my addr 172.21.15.50:0/310827801 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 -- 172.21.15.50:0/310827801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 msgr2=0x55c9a5cb5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 -- 172.21.15.50:0/310827801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 msgr2=0x55c9a5cc8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 -- 172.21.15.50:0/310827801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a59f8620 con 0x55c9a58f34d0 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76be36f640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f76bd36d640 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8640 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005760 tx=0x7f76ac0ec040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128afb127aa9f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.902+0000 7f769f7fe640 1 -- 172.21.15.50:0/310827801 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac102e60 con 0x55c9a58f34d0 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.903+0000 7f769f7fe640 1 -- 172.21.15.50:0/310827801 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76ac0ca340 con 0x55c9a58f34d0 2024-05-17T04:48:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.903+0000 7f769f7fe640 1 -- 172.21.15.50:0/310827801 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac049040 con 0x55c9a58f34d0 2024-05-17T04:48:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- 172.21.15.50:0/310827801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 msgr2=0x55c9a5cc8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8640 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005760 tx=0x7f76ac0ec040 comp rx=0 tx=0).stop 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- 172.21.15.50:0/310827801 shutdown_connections 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8640 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 --2- 172.21.15.50:0/310827801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cb5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- 172.21.15.50:0/310827801 >> 172.21.15.50:0/310827801 conn(0x55c9a5908300 msgr2=0x55c9a5cc11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- 172.21.15.50:0/310827801 shutdown_connections 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- 172.21.15.50:0/310827801 wait complete. 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 -- start start 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.904+0000 7f76c6d66740 1 Processor -- start 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81aa610 con 0x55c9a59141b0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81aba80 con 0x55c9a58eb2f0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76b81aace0 con 0x55c9a58f34d0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76c6d66740 10 client.0 _ll_register_callbacks cb 0x7f76a8009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76bdb6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:53326/0 (socket says 172.21.15.50:53326) 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 -- 172.21.15.50:0/2503376346 learned_addr learned my addr 172.21.15.50:0/2503376346 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76bd36d640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 -- 172.21.15.50:0/2503376346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 msgr2=0x55c9a5cc8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 -- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 msgr2=0x55c9a5cc8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76be36f640 1 -- 172.21.15.50:0/2503376346 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a5cc5bf0 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.905+0000 7f76bd36d640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76be36f640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaf063aeeca6c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b000fcf0 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f76b0005040 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ac0ca340 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76b0036050 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 10 client.16239 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9a5d3e2e0 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76ac0ca340 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f76b00194a0 con 0x55c9a59141b0 2024-05-17T04:48:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 10 client.16239 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 20 client.16239 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58a81c0 0x55c9a5d40b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 4 client.16239 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.906+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9a5cc0260 con 0x55c9a58a81c0 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.907+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58a81c0 0x55c9a5d40b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.907+0000 7f769d7fa640 10 client.16239 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.907+0000 7f76bdb6e640 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58a81c0 0x55c9a5d40b20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad340279ae8d6fc server_cookie=9bb70f342fb4adf2 in_seq=0 out_seq=0 2024-05-17T04:48:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.960+0000 7f769d7fa640 1 -- 172.21.15.50:0/2503376346 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c9a5cc0260 con 0x55c9a58a81c0 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.960+0000 7f769d7fa640 10 client.16239 handle_command_reply: tid=0 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.960+0000 7f76c6d66740 1 client.16239 shutdown 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 msgr2=0x55c9a5cc5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b000f770 comp rx=0 tx=0).stop 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 shutdown_connections 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c9a58a81c0 0x55c9a5d40b20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c9a59141b0 0x55c9a5cc5460 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9a58f34d0 0x55c9a5cc8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f769d7fa640 0 client.16239 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 --2- 172.21.15.50:0/2503376346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9a58eb2f0 0x55c9a5cc8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 >> 172.21.15.50:0/2503376346 conn(0x55c9a5908300 msgr2=0x55c9a5cb17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 shutdown_connections 2024-05-17T04:48:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 1 -- 172.21.15.50:0/2503376346 wait complete. 2024-05-17T04:48:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.961+0000 7f76c6d66740 20 client.16239 trim_cache size 0 max 16384 2024-05-17T04:48:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:48:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:48:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 msgr2=0x7f76a00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 0x7f76a00b4d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009fd0 tx=0x7f76a8005f90 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 msgr2=0x7f76b8199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009230 tx=0x7f76ac004040 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 shutdown_connections 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76b8103b00 0x7f76b81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f76a00b2860 0x7f76a00b4d70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b81031b0 0x7f76b8199d50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 --2- 172.21.15.50:0/3079355094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8102090 0x7f76b8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 >> 172.21.15.50:0/3079355094 conn(0x7f76b80fd8b0 msgr2=0x7f76b80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 shutdown_connections 2024-05-17T04:48:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:16.962+0000 7f76bfdd2640 1 -- 172.21.15.50:0/3079355094 wait complete. 2024-05-17T04:48:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 560.0 s, 121.2 tps, lat 875.327 ms stddev 242.783 2024-05-17T04:48:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:18 smithi105 ceph-mon[34795]: pgmap v798: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-05-17T04:48:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:18 smithi110 ceph-mon[35497]: pgmap v798: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-05-17T04:48:18.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:18 smithi050 ceph-mon[31739]: pgmap v798: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.7 MiB/s wr, 393 op/s 2024-05-17T04:48:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:20 smithi105 ceph-mon[34795]: pgmap v799: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-17T04:48:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:20 smithi110 ceph-mon[35497]: pgmap v799: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-17T04:48:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:20 smithi050 ceph-mon[31739]: pgmap v799: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-17T04:48:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:22 smithi105 ceph-mon[34795]: pgmap v800: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:48:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:22 smithi110 ceph-mon[35497]: pgmap v800: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:48:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:22 smithi050 ceph-mon[31739]: pgmap v800: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 2.9 MiB/s wr, 405 op/s 2024-05-17T04:48:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 565.0 s, 27.6 tps, lat 3782.259 ms stddev 1888.101 2024-05-17T04:48:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:24 smithi105 ceph-mon[34795]: pgmap v801: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-05-17T04:48:24.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:24 smithi110 ceph-mon[35497]: pgmap v801: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-05-17T04:48:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:24 smithi050 ceph-mon[31739]: pgmap v801: 129 pgs: 129 active+clean; 1.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-05-17T04:48:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:26 smithi105 ceph-mon[34795]: pgmap v802: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-05-17T04:48:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:26 smithi110 ceph-mon[35497]: pgmap v802: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-05-17T04:48:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:26 smithi050 ceph-mon[31739]: pgmap v802: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-05-17T04:48:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 570.0 s, 99.0 tps, lat 934.267 ms stddev 314.544 2024-05-17T04:48:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:28 smithi105 ceph-mon[34795]: pgmap v803: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 401 op/s 2024-05-17T04:48:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:28 smithi110 ceph-mon[35497]: pgmap v803: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 401 op/s 2024-05-17T04:48:28.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:28 smithi050 ceph-mon[31739]: pgmap v803: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 3.0 MiB/s wr, 401 op/s 2024-05-17T04:48:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/3457995914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01031b0 msgr2=0x7f9bd01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/3457995914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01031b0 0x7f9bd01035c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009900 tx=0x7f9bcc02a410 comp rx=0 tx=0).stop 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/3457995914 shutdown_connections 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/3457995914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 0x7f9bd010b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/3457995914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01042d0 0x7f9bd01046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/3457995914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01031b0 0x7f9bd01035c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/3457995914 >> 172.21.15.50:0/3457995914 conn(0x7f9bd00fe9b0 msgr2=0x7f9bd0100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/3457995914 shutdown_connections 2024-05-17T04:48:30.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.125+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/3457995914 wait complete. 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.126+0000 7f9bd7c5b640 1 Processor -- start 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 -- start start 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 0x7f9bd01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010eb90 con 0x7f9bd01042d0 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010ea10 con 0x7f9bd01031b0 2024-05-17T04:48:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd7c5b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd010ed10 con 0x7f9bd0104c20 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd51f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd59f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd51f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54372/0 (socket says 172.21.15.50:54372) 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd51f6640 1 -- 172.21.15.50:0/1411741269 learned_addr learned my addr 172.21.15.50:0/1411741269 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:57672/0 (socket says 172.21.15.50:57672) 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.127+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 0x7f9bd01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 -- 172.21.15.50:0/1411741269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 msgr2=0x7f9bd01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 0x7f9bd01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 -- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 msgr2=0x7f9bd0199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 -- 172.21.15.50:0/1411741269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01a1380 con 0x7f9bd01031b0 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.128+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009230 tx=0x7f9bcc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.129+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1411741269 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc035050 con 0x7f9bd01031b0 2024-05-17T04:48:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.129+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1411741269 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bcc03ab00 con 0x7f9bd01031b0 2024-05-17T04:48:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.129+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1411741269 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc003b80 con 0x7f9bd01031b0 2024-05-17T04:48:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.129+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd0074f70 con 0x7f9bd01031b0 2024-05-17T04:48:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.129+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd0075320 con 0x7f9bd01031b0 2024-05-17T04:48:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.132+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1411741269 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9bcc03a670 con 0x7f9bd01031b0 2024-05-17T04:48:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.132+0000 7f9bdebf1740 1 Processor -- start 2024-05-17T04:48:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 -- start start 2024-05-17T04:48:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44392f0 0x55f4c4805cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c480cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4777cc0 con 0x55f4c44621b0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4767860 con 0x55f4c44392f0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4c476b8a0 con 0x55f4c44414d0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54376/0 (socket says 172.21.15.50:54376) 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bd61f8640 1 -- 172.21.15.50:0/3435691601 learned_addr learned my addr 172.21.15.50:0/3435691601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.133+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c480cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44392f0 0x55f4c4805cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 -- 172.21.15.50:0/3435691601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 msgr2=0x55f4c480cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c480cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 -- 172.21.15.50:0/3435691601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44392f0 msgr2=0x55f4c4805cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44392f0 0x55f4c4805cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 -- 172.21.15.50:0/3435691601 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c472fd50 con 0x55f4c44621b0 2024-05-17T04:48:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bc6ffd640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 0x7f9bbc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1411741269 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9bcc1062f0 con 0x7f9bd01031b0 2024-05-17T04:48:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8010370 tx=0x7f9bc8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e62f36feb3129a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.134+0000 7f9bbbfff640 1 -- 172.21.15.50:0/3435691601 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8016070 con 0x55f4c44621b0 2024-05-17T04:48:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.135+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 0x7f9bbc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.135+0000 7f9bbbfff640 1 -- 172.21.15.50:0/3435691601 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc8004d20 con 0x55f4c44621b0 2024-05-17T04:48:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.135+0000 7f9bbbfff640 1 -- 172.21.15.50:0/3435691601 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80052c0 con 0x55f4c44621b0 2024-05-17T04:48:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bdebf1740 1 -- 172.21.15.50:0/3435691601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 msgr2=0x55f4c4805770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bd59f7640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 0x7f9bbc0b4e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0c95a0 tx=0x7f9bcc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8010370 tx=0x7f9bc8010830 comp rx=0 tx=0).stop 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bdebf1740 1 -- 172.21.15.50:0/3435691601 shutdown_connections 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c480cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.136+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44392f0 0x55f4c4805cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/3435691601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44621b0 0x55f4c4805770 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 -- 172.21.15.50:0/3435691601 >> 172.21.15.50:0/3435691601 conn(0x55f4c4456300 msgr2=0x55f4c48011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 -- 172.21.15.50:0/3435691601 shutdown_connections 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 -- 172.21.15.50:0/3435691601 wait complete. 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 -- start start 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.137+0000 7f9bdebf1740 1 Processor -- start 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44392f0 0x55f4c480f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f4c46a3ab0 con 0x55f4c44414d0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4c47244d0 con 0x55f4c44621b0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54378/0 (socket says 172.21.15.50:54378) 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 -- 172.21.15.50:0/1867362825 learned_addr learned my addr 172.21.15.50:0/1867362825 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4c480d690 con 0x55f4c44392f0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44392f0 0x55f4c480f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bdebf1740 10 client.0 _ll_register_callbacks cb 0x55f4c4762950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 -- 172.21.15.50:0/1867362825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44392f0 msgr2=0x55f4c480f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44392f0 0x55f4c480f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 -- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 msgr2=0x55f4c4816530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd51f6640 1 -- 172.21.15.50:0/1867362825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c4725340 con 0x55f4c44414d0 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.138+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.139+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0004070 tx=0x7f9bc0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753d729555676038 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.139+0000 7f9bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.139+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0021410 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.139+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9bc00215b0 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.139+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc8004d20 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.140+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bc002c050 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.140+0000 7f9bdebf1740 10 client.16251 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.140+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4c44cd790 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.141+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc8004d20 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9bc0020710 con 0x55f4c44414d0 2024-05-17T04:48:30.144 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bdebf1740 10 client.16251 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bdebf1740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bdebf1740 20 client.16251 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c4420d80 0x55f4c481ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.142+0000 7f9bdebf1740 4 client.16251 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.143+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c4779c10 con 0x55f4c4420d80 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.143+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c4420d80 0x55f4c481ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.143+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c4420d80 0x55f4c481ccf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea1d18d09031103 server_cookie=931643dbe92b9403 in_seq=0 out_seq=0 2024-05-17T04:48:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.143+0000 7f9bb97fa640 10 client.16251 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.145+0000 7f9bb97fa640 1 -- 172.21.15.50:0/1867362825 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4c4779c10 con 0x55f4c4420d80 2024-05-17T04:48:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.145+0000 7f9bb97fa640 10 client.16251 handle_command_reply: tid=0 2024-05-17T04:48:30.147 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 client.16251 shutdown 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 msgr2=0x55f4c480f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0004070 tx=0x7f9bc0004530 comp rx=0 tx=0).stop 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 shutdown_connections 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c4420d80 0x55f4c481ccf0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44414d0 0x55f4c480f5d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/1867362825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44392f0 0x55f4c480f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 >> 172.21.15.50:0/1867362825 conn(0x55f4c4456300 msgr2=0x55f4c48001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bb97fa640 0 client.16251 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.146+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 shutdown_connections 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.147+0000 7f9bdebf1740 1 -- 172.21.15.50:0/1867362825 wait complete. 2024-05-17T04:48:30.148 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.147+0000 7f9bdebf1740 20 client.16251 trim_cache size 0 max 16384 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.176+0000 7f9bdebf1740 1 Processor -- start 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 -- start start 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c4816450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ac6f0 con 0x55f4c44392f0 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01a9fc0 con 0x55f4c44621b0 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01a7e10 con 0x55f4c44414d0 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54384/0 (socket says 172.21.15.50:54384) 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bd61f8640 1 -- 172.21.15.50:0/4129895164 learned_addr learned my addr 172.21.15.50:0/4129895164 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c4816450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.177+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 -- 172.21.15.50:0/4129895164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 msgr2=0x55f4c4816990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 -- 172.21.15.50:0/4129895164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 msgr2=0x55f4c4808bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 -- 172.21.15.50:0/4129895164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c4546620 con 0x55f4c44414d0 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c4816450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0020ae0 tx=0x7f9bc0020f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d2af15d47f355f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bbb7fe640 1 -- 172.21.15.50:0/4129895164 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0034070 con 0x55f4c44414d0 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bbb7fe640 1 -- 172.21.15.50:0/4129895164 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc0020800 con 0x55f4c44414d0 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.178+0000 7f9bbb7fe640 1 -- 172.21.15.50:0/4129895164 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00055f0 con 0x55f4c44414d0 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.179+0000 7f9bdebf1740 1 -- 172.21.15.50:0/4129895164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 msgr2=0x55f4c4816450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.179+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c4816450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0020ae0 tx=0x7f9bc0020f70 comp rx=0 tx=0).stop 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 -- 172.21.15.50:0/4129895164 shutdown_connections 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44621b0 0x55f4c4816990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44414d0 0x55f4c4816450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/4129895164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c4808bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 -- 172.21.15.50:0/4129895164 >> 172.21.15.50:0/4129895164 conn(0x55f4c4456300 msgr2=0x55f4c48001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 -- 172.21.15.50:0/4129895164 shutdown_connections 2024-05-17T04:48:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 -- 172.21.15.50:0/4129895164 wait complete. 2024-05-17T04:48:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 -- start start 2024-05-17T04:48:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.180+0000 7f9bdebf1740 1 Processor -- start 2024-05-17T04:48:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44414d0 0x55f4c48159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44621b0 0x55f4c4815f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01a8480 con 0x55f4c44392f0 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01a98f0 con 0x55f4c44414d0 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01a8b50 con 0x55f4c44621b0 2024-05-17T04:48:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bdebf1740 10 client.0 _ll_register_callbacks cb 0x7f9bc000be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd61f8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54390/0 (socket says 172.21.15.50:54390) 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd61f8640 1 -- 172.21.15.50:0/2872941449 learned_addr learned my addr 172.21.15.50:0/2872941449 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44621b0 0x55f4c4815f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44414d0 0x55f4c48159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.182+0000 7f9bd61f8640 1 -- 172.21.15.50:0/2872941449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44621b0 msgr2=0x55f4c4815f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44621b0 0x55f4c4815f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bd61f8640 1 -- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44414d0 msgr2=0x55f4c48159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44414d0 0x55f4c48159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bd61f8640 1 -- 172.21.15.50:0/2872941449 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c4882f70 con 0x55f4c44392f0 2024-05-17T04:48:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bd61f8640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004c10 tx=0x7f9bc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd1f470a2cbbd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8016030 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9bc802b580 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.183+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc0020800 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.184+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bc8024040 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.184+0000 7f9bdebf1740 10 client.16257 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.184+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4c4884170 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc0020800 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9bc802a040 con 0x55f4c44392f0 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 10 client.16257 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 20 client.16257 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c43f61c0 0x55f4c4886a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 4 client.16257 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:48:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4c47733f0 con 0x55f4c43f61c0 2024-05-17T04:48:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.185+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c43f61c0 0x55f4c4886a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.186+0000 7f9bd51f6640 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c43f61c0 0x55f4c4886a50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d74d2ff2bbfd75a server_cookie=2761f5cf031aa688 in_seq=0 out_seq=0 2024-05-17T04:48:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.186+0000 7f9bb97fa640 10 client.16257 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.237+0000 7f9bb97fa640 1 -- 172.21.15.50:0/2872941449 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4c47733f0 con 0x55f4c43f61c0 2024-05-17T04:48:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.237+0000 7f9bb97fa640 10 client.16257 handle_command_reply: tid=0 2024-05-17T04:48:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 client.16257 shutdown 2024-05-17T04:48:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 msgr2=0x55f4c48154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004c10 tx=0x7f9bc8009bb0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 shutdown_connections 2024-05-17T04:48:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4c44621b0 0x55f4c4815f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4c44414d0 0x55f4c48159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f4c43f61c0 0x55f4c4886a50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 --2- 172.21.15.50:0/2872941449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f4c44392f0 0x55f4c48154b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.238+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 >> 172.21.15.50:0/2872941449 conn(0x55f4c4456300 msgr2=0x55f4c47fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.239+0000 7f9bb97fa640 0 client.16257 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.239+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 shutdown_connections 2024-05-17T04:48:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.239+0000 7f9bdebf1740 1 -- 172.21.15.50:0/2872941449 wait complete. 2024-05-17T04:48:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.239+0000 7f9bdebf1740 20 client.16257 trim_cache size 0 max 16384 2024-05-17T04:48:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:48:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:48:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.240+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 msgr2=0x7f9bbc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.240+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 0x7f9bbc0b4e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0c95a0 tx=0x7f9bcc031400 comp rx=0 tx=0).stop 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.240+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 msgr2=0x7f9bd0199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.240+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009230 tx=0x7f9bcc004270 comp rx=0 tx=0).stop 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 shutdown_connections 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bd0104c20 0x7f9bd01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9bbc0b2930 0x7f9bbc0b4e40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bd01042d0 0x7f9bd0199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 --2- 172.21.15.50:0/1411741269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bd01031b0 0x7f9bd0199840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 >> 172.21.15.50:0/1411741269 conn(0x7f9bd00fe9b0 msgr2=0x7f9bd01075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 shutdown_connections 2024-05-17T04:48:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:30.241+0000 7f9bd7c5b640 1 -- 172.21.15.50:0/1411741269 wait complete. 2024-05-17T04:48:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:30 smithi105 ceph-mon[34795]: pgmap v804: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-05-17T04:48:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:30 smithi110 ceph-mon[35497]: pgmap v804: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-05-17T04:48:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:30 smithi050 ceph-mon[31739]: pgmap v804: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-05-17T04:48:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:32 smithi105 ceph-mon[34795]: pgmap v805: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-05-17T04:48:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:32 smithi110 ceph-mon[35497]: pgmap v805: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-05-17T04:48:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:32 smithi050 ceph-mon[31739]: pgmap v805: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-05-17T04:48:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 575.0 s, 107.2 tps, lat 930.189 ms stddev 228.204 2024-05-17T04:48:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:34 smithi105 ceph-mon[34795]: pgmap v806: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-05-17T04:48:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:34 smithi110 ceph-mon[35497]: pgmap v806: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-05-17T04:48:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:34 smithi050 ceph-mon[31739]: pgmap v806: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-05-17T04:48:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:36 smithi105 ceph-mon[34795]: pgmap v807: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-05-17T04:48:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:36 smithi110 ceph-mon[35497]: pgmap v807: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-05-17T04:48:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:36 smithi050 ceph-mon[31739]: pgmap v807: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-05-17T04:48:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 580.0 s, 118.4 tps, lat 850.850 ms stddev 270.135 2024-05-17T04:48:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:38 smithi105 ceph-mon[34795]: pgmap v808: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-05-17T04:48:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:38 smithi110 ceph-mon[35497]: pgmap v808: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-05-17T04:48:38.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:38 smithi050 ceph-mon[31739]: pgmap v808: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.3 MiB/s wr, 426 op/s 2024-05-17T04:48:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:40 smithi105 ceph-mon[34795]: pgmap v809: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-17T04:48:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:40 smithi110 ceph-mon[35497]: pgmap v809: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-17T04:48:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:40 smithi050 ceph-mon[31739]: pgmap v809: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-17T04:48:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:42 smithi105 ceph-mon[34795]: pgmap v810: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 378 op/s 2024-05-17T04:48:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:42 smithi110 ceph-mon[35497]: pgmap v810: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 378 op/s 2024-05-17T04:48:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:42 smithi050 ceph-mon[31739]: pgmap v810: 129 pgs: 129 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.8 MiB/s wr, 378 op/s 2024-05-17T04:48:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 585.0 s, 113.2 tps, lat 875.095 ms stddev 199.856 2024-05-17T04:48:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 -- 172.21.15.50:0/2215614996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc104c00 msgr2=0x7f26fc10b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 --2- 172.21.15.50:0/2215614996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc104c00 0x7f26fc10b500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009be0 tx=0x7f26f402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 -- 172.21.15.50:0/2215614996 shutdown_connections 2024-05-17T04:48:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 --2- 172.21.15.50:0/2215614996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc104c00 0x7f26fc10b500 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 --2- 172.21.15.50:0/2215614996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc1042b0 0x7f26fc1046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 --2- 172.21.15.50:0/2215614996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 0x7f26fc1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 -- 172.21.15.50:0/2215614996 >> 172.21.15.50:0/2215614996 conn(0x7f26fc0fe9b0 msgr2=0x7f26fc100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 -- 172.21.15.50:0/2215614996 shutdown_connections 2024-05-17T04:48:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.404+0000 7f270357a640 1 -- 172.21.15.50:0/2215614996 wait complete. 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 Processor -- start 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 -- start start 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 0x7f26fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104c00 0x7f26fc0755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc10eb70 con 0x7f26fc1042b0 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc10e9f0 con 0x7f26fc104c00 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f270357a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc10ecf0 con 0x7f26fc103190 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f2700b15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f2700b15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49588/0 (socket says 172.21.15.50:49588) 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f2700b15640 1 -- 172.21.15.50:0/1290785237 learned_addr learned my addr 172.21.15.50:0/1290785237 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f2701316640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 0x7f26fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.405+0000 7f2701b17640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104c00 0x7f26fc0755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 -- 172.21.15.50:0/1290785237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 msgr2=0x7f26fc0769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 0x7f26fc0769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 -- 172.21.15.50:0/1290785237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104c00 msgr2=0x7f26fc0755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104c00 0x7f26fc0755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 -- 172.21.15.50:0/1290785237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc075d00 con 0x7f26fc1042b0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f2700b15640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9c0 tx=0x7f26ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f26f27fc640 1 -- 172.21.15.50:0/1290785237 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00ccc0 con 0x7f26fc1042b0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f26f27fc640 1 -- 172.21.15.50:0/1290785237 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26ec0054b0 con 0x7f26fc1042b0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc1a3830 con 0x7f26fc1042b0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f26f27fc640 1 -- 172.21.15.50:0/1290785237 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec010430 con 0x7f26fc1042b0 2024-05-17T04:48:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.406+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a3d20 con 0x7f26fc1042b0 2024-05-17T04:48:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 Processor -- start 2024-05-17T04:48:43.411 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 -- start start 2024-05-17T04:48:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431ade8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56431ad51cc0 con 0x56431aa3c1b0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56431ad41860 con 0x56431aa1b4d0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f270a435740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56431ad458a0 con 0x56431aa132f0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f26f27fc640 1 -- 172.21.15.50:0/1290785237 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26ec0105d0 con 0x7f26fc1042b0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f2701b17640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f2701b17640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49604/0 (socket says 172.21.15.50:49604) 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.411+0000 7f2701b17640 1 -- 172.21.15.50:0/2325405563 learned_addr learned my addr 172.21.15.50:0/2325405563 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2700b15640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701316640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431ade8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701b17640 1 -- 172.21.15.50:0/2325405563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 msgr2=0x56431ade8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701b17640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431ade8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701b17640 1 -- 172.21.15.50:0/2325405563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 msgr2=0x56431adef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701b17640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701b17640 1 -- 172.21.15.50:0/2325405563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431ad09d50 con 0x56431aa3c1b0 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.412+0000 7f2701316640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431ade8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.413+0000 7f2701b17640 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f26f402acb0 tx=0x7f26f4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1577e1ac0d229764 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.413+0000 7f26f27fc640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 0x7f26e80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.413+0000 7f26f27fc640 1 -- 172.21.15.50:0/1290785237 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f26ec0e29b0 con 0x7f26fc1042b0 2024-05-17T04:48:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.413+0000 7f26e77fe640 1 -- 172.21.15.50:0/2325405563 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4041030 con 0x56431aa3c1b0 2024-05-17T04:48:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.414+0000 7f2701316640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 0x7f26e80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.414+0000 7f26e77fe640 1 -- 172.21.15.50:0/2325405563 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26f4037040 con 0x56431aa3c1b0 2024-05-17T04:48:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.414+0000 7f26e77fe640 1 -- 172.21.15.50:0/2325405563 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4036040 con 0x56431aa3c1b0 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 -- 172.21.15.50:0/2325405563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 msgr2=0x56431ade7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f26f402acb0 tx=0x7f26f4005c50 comp rx=0 tx=0).stop 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 -- 172.21.15.50:0/2325405563 shutdown_connections 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431ade8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 --2- 172.21.15.50:0/2325405563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431ade7f80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 -- 172.21.15.50:0/2325405563 >> 172.21.15.50:0/2325405563 conn(0x56431aa30300 msgr2=0x56431addb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f2701316640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 0x7f26e80b4df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56431ade9380 tx=0x7f26f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 -- 172.21.15.50:0/2325405563 shutdown_connections 2024-05-17T04:48:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.415+0000 7f270a435740 1 -- 172.21.15.50:0/2325405563 wait complete. 2024-05-17T04:48:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.416+0000 7f270a435740 1 -- start start 2024-05-17T04:48:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.416+0000 7f270a435740 1 Processor -- start 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.417+0000 7f270a435740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.417+0000 7f2701316640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.417+0000 7f2701316640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58340/0 (socket says 172.21.15.50:58340) 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56431ac7dab0 con 0x56431aa3c1b0 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56431acfe4d0 con 0x56431aa1b4d0 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56431adefe40 con 0x56431aa132f0 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 -- 172.21.15.50:0/4151148966 learned_addr learned my addr 172.21.15.50:0/4151148966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701b17640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 10 client.0 _ll_register_callbacks cb 0x56431ad3c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2700b15640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 -- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 msgr2=0x56431adf1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 -- 172.21.15.50:0/4151148966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 msgr2=0x56431adf8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 -- 172.21.15.50:0/4151148966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431acff340 con 0x56431aa132f0 2024-05-17T04:48:43.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f2701316640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009730 tx=0x7f26f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c317765f666140e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f26e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8024070 con 0x56431aa132f0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f26f80108d0 con 0x56431aa132f0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.419+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f4037040 con 0x56431aa132f0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.420+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26f8030050 con 0x56431aa132f0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.420+0000 7f270a435740 10 client.25999 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.420+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56431aaa7790 con 0x56431aa132f0 2024-05-17T04:48:43.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.420+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26f4037040 con 0x56431aa132f0 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.421+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f26f8003b60 con 0x56431aa132f0 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 10 client.25999 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 20 client.25999 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9fad80 0x56431adff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 4 client.25999 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:48:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.422+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56431ad53c10 con 0x56431a9fad80 2024-05-17T04:48:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.423+0000 7f2700b15640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9fad80 0x56431adff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.423+0000 7f26e4ff9640 10 client.25999 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.423+0000 7f2700b15640 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9fad80 0x56431adff510 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fef1953f52e0b server_cookie=8a318e774e3f3ee in_seq=0 out_seq=0 2024-05-17T04:48:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.424+0000 7f26e4ff9640 1 -- 172.21.15.50:0/4151148966 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56431ad53c10 con 0x56431a9fad80 2024-05-17T04:48:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.424+0000 7f26e4ff9640 10 client.25999 handle_command_reply: tid=0 2024-05-17T04:48:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.424+0000 7f270a435740 1 client.25999 shutdown 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 msgr2=0x56431adf19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009730 tx=0x7f26f80046c0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 shutdown_connections 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9fad80 0x56431adff510 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 --2- 172.21.15.50:0/4151148966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa132f0 0x56431adf19a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 >> 172.21.15.50:0/4151148966 conn(0x56431aa30300 msgr2=0x56431ade6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f26e4ff9640 0 client.25999 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 shutdown_connections 2024-05-17T04:48:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.425+0000 7f270a435740 1 -- 172.21.15.50:0/4151148966 wait complete. 2024-05-17T04:48:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.426+0000 7f270a435740 20 client.25999 trim_cache size 0 max 16384 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.450+0000 7f270a435740 1 Processor -- start 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.450+0000 7f270a435740 1 -- start start 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa1b4d0 0x56431ade6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ac780 con 0x56431aa3c1b0 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1aa050 con 0x56431aa132f0 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f270a435740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1a7ec0 con 0x56431aa1b4d0 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40548/0 (socket says 172.21.15.50:40548) 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 -- 172.21.15.50:0/811653926 learned_addr learned my addr 172.21.15.50:0/811653926 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701b17640 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2700b15640 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa1b4d0 0x56431ade6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 -- 172.21.15.50:0/811653926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa1b4d0 msgr2=0x56431ade6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa1b4d0 0x56431ade6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 -- 172.21.15.50:0/811653926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 msgr2=0x56431adf8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.451+0000 7f2701316640 1 -- 172.21.15.50:0/811653926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431ab20620 con 0x56431aa132f0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.452+0000 7f2701316640 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26f80107a0 tx=0x7f26f8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e42a12597bad0be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.452+0000 7f26e6ffd640 1 -- 172.21.15.50:0/811653926 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8024070 con 0x56431aa132f0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.452+0000 7f26e6ffd640 1 -- 172.21.15.50:0/811653926 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26f8039070 con 0x56431aa132f0 2024-05-17T04:48:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.452+0000 7f26e6ffd640 1 -- 172.21.15.50:0/811653926 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8010b80 con 0x56431aa132f0 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 -- 172.21.15.50:0/811653926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 msgr2=0x56431adec550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26f80107a0 tx=0x7f26f8005740 comp rx=0 tx=0).stop 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 -- 172.21.15.50:0/811653926 shutdown_connections 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa3c1b0 0x56431adf8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa1b4d0 0x56431ade6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 --2- 172.21.15.50:0/811653926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa132f0 0x56431adec550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 -- 172.21.15.50:0/811653926 >> 172.21.15.50:0/811653926 conn(0x56431aa30300 msgr2=0x56431ade6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 -- 172.21.15.50:0/811653926 shutdown_connections 2024-05-17T04:48:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.454+0000 7f270a435740 1 -- 172.21.15.50:0/811653926 wait complete. 2024-05-17T04:48:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.455+0000 7f270a435740 1 -- start start 2024-05-17T04:48:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.455+0000 7f270a435740 1 Processor -- start 2024-05-17T04:48:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.455+0000 7f270a435740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa132f0 0x56431adf9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1a8510 con 0x56431aa132f0 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1a9980 con 0x56431aa1b4d0 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1a8be0 con 0x56431aa3c1b0 2024-05-17T04:48:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f270a435740 10 client.0 _ll_register_callbacks cb 0x7f26f8009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2700b15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701316640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa132f0 0x56431adf9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:58354/0 (socket says 172.21.15.50:58354) 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 -- 172.21.15.50:0/1572949583 learned_addr learned my addr 172.21.15.50:0/1572949583 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 -- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 msgr2=0x56431adf7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 -- 172.21.15.50:0/1572949583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa132f0 msgr2=0x56431adf9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa132f0 0x56431adf9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.456+0000 7f2701b17640 1 -- 172.21.15.50:0/1572949583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431adf8650 con 0x56431aa3c1b0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f2701b17640 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26f40092f0 tx=0x7f26f402a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6702e01641d156 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f26e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4036050 con 0x56431aa3c1b0 2024-05-17T04:48:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f26f4003970 con 0x56431aa3c1b0 2024-05-17T04:48:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f8039070 con 0x56431aa3c1b0 2024-05-17T04:48:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26f4041040 con 0x56431aa3c1b0 2024-05-17T04:48:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.457+0000 7f270a435740 10 client.26005 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56431ae666f0 con 0x56431aa3c1b0 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26f4037040 con 0x56431aa3c1b0 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f26f403c040 con 0x56431aa3c1b0 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 10 client.26005 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 10 client.26005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 20 client.26005 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9d01c0 0x56431ae68c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 4 client.26005 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:48:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f26f8039070 con 0x56431a9d01c0 2024-05-17T04:48:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.458+0000 7f2701316640 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9d01c0 0x56431ae68c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.459+0000 7f26e4ff9640 10 client.26005 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.459+0000 7f2701316640 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9d01c0 0x56431ae68c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e2bacdf1242d34 server_cookie=fb5ab426126b58f2 in_seq=0 out_seq=0 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f26e4ff9640 1 -- 172.21.15.50:0/1572949583 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f26f8039070 con 0x56431a9d01c0 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f26e4ff9640 10 client.26005 handle_command_reply: tid=0 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 client.26005 shutdown 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 msgr2=0x56431adf7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26f40092f0 tx=0x7f26f402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 shutdown_connections 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56431aa3c1b0 0x56431adf7fe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431aa1b4d0 0x56431adf7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.497+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56431a9d01c0 0x56431ae68c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f26e4ff9640 0 client.26005 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f270a435740 1 --2- 172.21.15.50:0/1572949583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56431aa132f0 0x56431adf9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 >> 172.21.15.50:0/1572949583 conn(0x56431aa30300 msgr2=0x56431adeb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 shutdown_connections 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f270a435740 1 -- 172.21.15.50:0/1572949583 wait complete. 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.498+0000 7f270a435740 20 client.26005 trim_cache size 0 max 16384 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:48:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 msgr2=0x7f26e80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 0x7f26e80b4df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x56431ade9380 tx=0x7f26f8005f90 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 msgr2=0x7f26fc075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9c0 tx=0x7f26ec00ee80 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 shutdown_connections 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104c00 0x7f26fc0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f26e80b28e0 0x7f26e80b4df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26fc1042b0 0x7f26fc075090 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 --2- 172.21.15.50:0/1290785237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26fc103190 0x7f26fc0769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 >> 172.21.15.50:0/1290785237 conn(0x7f26fc0fe9b0 msgr2=0x7f26fc107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 shutdown_connections 2024-05-17T04:48:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:43.499+0000 7f270357a640 1 -- 172.21.15.50:0/1290785237 wait complete. 2024-05-17T04:48:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:44 smithi105 ceph-mon[34795]: pgmap v811: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 404 op/s 2024-05-17T04:48:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:44 smithi110 ceph-mon[35497]: pgmap v811: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 404 op/s 2024-05-17T04:48:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:44 smithi050 ceph-mon[31739]: pgmap v811: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 404 op/s 2024-05-17T04:48:45.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:45 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:48:45.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:45 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:48:45.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:45 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:48:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:46 smithi105 ceph-mon[34795]: pgmap v812: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-17T04:48:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:46 smithi110 ceph-mon[35497]: pgmap v812: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-17T04:48:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:46 smithi050 ceph-mon[31739]: pgmap v812: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-17T04:48:47.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:48:47.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:48:47.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:48:47.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:48:47.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:48:47.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:48:47.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:48:47.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:48:47.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:48:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 590.0 s, 141.0 tps, lat 720.542 ms stddev 291.651 2024-05-17T04:48:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:48 smithi105 ceph-mon[34795]: pgmap v813: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-17T04:48:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:48 smithi110 ceph-mon[35497]: pgmap v813: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-17T04:48:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:48 smithi050 ceph-mon[31739]: pgmap v813: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 3.1 MiB/s wr, 429 op/s 2024-05-17T04:48:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:50 smithi105 ceph-mon[34795]: pgmap v814: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2024-05-17T04:48:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:50 smithi110 ceph-mon[35497]: pgmap v814: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2024-05-17T04:48:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:50 smithi050 ceph-mon[31739]: pgmap v814: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.0 MiB/s wr, 410 op/s 2024-05-17T04:48:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:52 smithi105 ceph-mon[34795]: pgmap v815: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-05-17T04:48:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:52 smithi110 ceph-mon[35497]: pgmap v815: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-05-17T04:48:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:52 smithi050 ceph-mon[31739]: pgmap v815: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 3.9 MiB/s wr, 474 op/s 2024-05-17T04:48:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 595.0 s, 145.6 tps, lat 701.051 ms stddev 218.486 2024-05-17T04:48:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:54 smithi105 ceph-mon[34795]: pgmap v816: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-17T04:48:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:54 smithi110 ceph-mon[35497]: pgmap v816: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-17T04:48:54.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:54 smithi050 ceph-mon[31739]: pgmap v816: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-17T04:48:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:56 smithi105 ceph-mon[34795]: pgmap v817: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-05-17T04:48:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:56 smithi110 ceph-mon[35497]: pgmap v817: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-05-17T04:48:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:56 smithi050 ceph-mon[31739]: pgmap v817: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-05-17T04:48:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 -- 172.21.15.50:0/3419277403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 msgr2=0x7f8ecc112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3419277403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc112a40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ca0 tx=0x7f8ec402a950 comp rx=0 tx=0).stop 2024-05-17T04:48:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 -- 172.21.15.50:0/3419277403 shutdown_connections 2024-05-17T04:48:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3419277403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc112a40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3419277403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 0x7f8ecc110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3419277403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc10d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 -- 172.21.15.50:0/3419277403 >> 172.21.15.50:0/3419277403 conn(0x7f8ecc0701c0 msgr2=0x7f8ecc072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 -- 172.21.15.50:0/3419277403 shutdown_connections 2024-05-17T04:48:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.645+0000 7f8ed408f640 1 -- 172.21.15.50:0/3419277403 wait complete. 2024-05-17T04:48:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 Processor -- start 2024-05-17T04:48:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 -- start start 2024-05-17T04:48:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 0x7f8ecc1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc1ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed408f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc113e20 con 0x7f8ecc110590 2024-05-17T04:48:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44208/0 (socket says 172.21.15.50:44208) 2024-05-17T04:48:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/3889302194 learned_addr learned my addr 172.21.15.50:0/3889302194 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed162a640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 0x7f8ecc1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.646+0000 7f8ed262c640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc1ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc113ca0 con 0x7f8ecc10dc30 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc113fa0 con 0x7f8ecc109f20 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 -- 172.21.15.50:0/3889302194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 msgr2=0x7f8ecc1a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 -- 172.21.15.50:0/3889302194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 msgr2=0x7f8ecc1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 0x7f8ecc1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 -- 172.21.15.50:0/3889302194 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1abcf0 con 0x7f8ecc110590 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ed262c640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc1ab680 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402ae70 tx=0x7f8ec4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ec2ffd640 1 -- 172.21.15.50:0/3889302194 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4034040 con 0x7f8ecc110590 2024-05-17T04:48:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.647+0000 7f8ec2ffd640 1 -- 172.21.15.50:0/3889302194 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec402f960 con 0x7f8ecc110590 2024-05-17T04:48:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.648+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1abf80 con 0x7f8ecc110590 2024-05-17T04:48:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.648+0000 7f8ec2ffd640 1 -- 172.21.15.50:0/3889302194 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004530 con 0x7f8ecc110590 2024-05-17T04:48:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.648+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1ac390 con 0x7f8ecc110590 2024-05-17T04:48:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 Processor -- start 2024-05-17T04:48:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ec2ffd640 1 -- 172.21.15.50:0/3889302194 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8ec4037020 con 0x7f8ecc110590 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 -- start start 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0f71b0 0x561aef49a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561aef40ccc0 con 0x561aef0d64d0 2024-05-17T04:48:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561aef3fc860 con 0x561aef0f71b0 2024-05-17T04:48:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8edb026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561aef4008a0 con 0x561aef0ce2f0 2024-05-17T04:48:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ed162a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0f71b0 0x561aef49a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44216/0 (socket says 172.21.15.50:44216) 2024-05-17T04:48:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/2785444600 learned_addr learned my addr 172.21.15.50:0/2785444600 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.651+0000 7f8ed162a640 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/2785444600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0f71b0 msgr2=0x561aef49a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0f71b0 0x561aef49a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/2785444600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 msgr2=0x561aef4a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/2785444600 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aef3c4d50 con 0x561aef0ce2f0 2024-05-17T04:48:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ec2ffd640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 0x7f8eb80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80115c0 tx=0x7f8ec8011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77134f954d87a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ed162a640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 0x7f8eb80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8ec2ffd640 1 -- 172.21.15.50:0/3889302194 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8ec41032f0 con 0x7f8ecc110590 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8eb7fff640 1 -- 172.21.15.50:0/2785444600 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8005020 con 0x561aef0ce2f0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8eb7fff640 1 -- 172.21.15.50:0/2785444600 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec80047a0 con 0x561aef0ce2f0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.652+0000 7f8eb7fff640 1 -- 172.21.15.50:0/2785444600 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800f7e0 con 0x561aef0ce2f0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8ed162a640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 0x7f8eb80b4cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x561aef49b960 tx=0x7f8ebc004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- 172.21.15.50:0/2785444600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 msgr2=0x561aef49acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80115c0 tx=0x7f8ec8011a80 comp rx=0 tx=0).stop 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- 172.21.15.50:0/2785444600 shutdown_connections 2024-05-17T04:48:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef49acf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 --2- 172.21.15.50:0/2785444600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0f71b0 0x561aef49a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- 172.21.15.50:0/2785444600 >> 172.21.15.50:0/2785444600 conn(0x561aef0eb300 msgr2=0x561aef4961a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- 172.21.15.50:0/2785444600 shutdown_connections 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- 172.21.15.50:0/2785444600 wait complete. 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.653+0000 7f8edb026740 1 -- start start 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 Processor -- start 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 0x561aef4a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0f71b0 0x561aef4ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44220/0 (socket says 172.21.15.50:44220) 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561aef338ab0 con 0x561aef0f71b0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/1993304643 learned_addr learned my addr 172.21.15.50:0/1993304643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8ed162a640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 0x561aef4a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8ed262c640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0f71b0 0x561aef4ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561aef3b94d0 con 0x561aef0d64d0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561aef4a2690 con 0x561aef0ce2f0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.654+0000 7f8edb026740 10 client.0 _ll_register_callbacks cb 0x561aef3f7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 msgr2=0x561aef4a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 0x561aef4a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/1993304643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0f71b0 msgr2=0x561aef4ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0f71b0 0x561aef4ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aef3ba340 con 0x561aef0ce2f0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed162a640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 0x561aef4a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80058c0 tx=0x7f8ec80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe36484942b258a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec801d050 con 0x561aef0ce2f0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8ec8005210 con 0x561aef0ce2f0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec80047a0 con 0x561aef0ce2f0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8032050 con 0x561aef0ce2f0 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8edb026740 10 client.26017 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561aef162790 con 0x561aef0ce2f0 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.655+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec80047a0 con 0x561aef0ce2f0 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec8003c40 con 0x561aef0ce2f0 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 10 client.26017 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 20 client.26017 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef0b5d80 0x561aef4b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 4 client.26017 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aef40ec10 con 0x561aef0b5d80 2024-05-17T04:48:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.656+0000 7f8ed162a640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef0b5d80 0x561aef4b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8eb57fa640 10 client.26017 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8ed162a640 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef0b5d80 0x561aef4b15e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d4c1bdaea492f8 server_cookie=99f5291674e6cd78 in_seq=0 out_seq=0 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8eb57fa640 1 -- 172.21.15.50:0/1993304643 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561aef40ec10 con 0x561aef0b5d80 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8eb57fa640 10 client.26017 handle_command_reply: tid=0 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 client.26017 shutdown 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 msgr2=0x561aef4a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80058c0 tx=0x7f8ec80042f0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 shutdown_connections 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef0b5d80 0x561aef4b15e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0f71b0 0x561aef4ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8eb57fa640 0 client.26017 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0d64d0 0x561aef4a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 --2- 172.21.15.50:0/1993304643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0ce2f0 0x561aef4a4030 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.657+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 >> 172.21.15.50:0/1993304643 conn(0x561aef0eb300 msgr2=0x561aef499010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.658+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 shutdown_connections 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.658+0000 7f8edb026740 1 -- 172.21.15.50:0/1993304643 wait complete. 2024-05-17T04:48:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.658+0000 7f8edb026740 20 client.26017 trim_cache size 0 max 16384 2024-05-17T04:48:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.676+0000 7f8edb026740 1 Processor -- start 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.676+0000 7f8edb026740 1 -- start start 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b9210 con 0x561aef0d64d0 2024-05-17T04:48:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b6ae0 con 0x561aef0ce2f0 2024-05-17T04:48:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8edb026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b4930 con 0x561aef0f71b0 2024-05-17T04:48:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57256/0 (socket says 172.21.15.50:57256) 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 -- 172.21.15.50:0/2949909180 learned_addr learned my addr 172.21.15.50:0/2949909180 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40116/0 (socket says 172.21.15.50:40116) 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 -- 172.21.15.50:0/2949909180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 msgr2=0x561aef4ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 -- 172.21.15.50:0/2949909180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 msgr2=0x561aef49eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed162a640 1 -- 172.21.15.50:0/2949909180 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aef1db620 con 0x561aef0d64d0 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.677+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.678+0000 7f8ed162a640 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x561aef4b1d40 tx=0x7f8ebc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0724b785dabf10c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.678+0000 7f8eb77fe640 1 -- 172.21.15.50:0/2949909180 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc011d10 con 0x561aef0d64d0 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.678+0000 7f8eb77fe640 1 -- 172.21.15.50:0/2949909180 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ebc0054b0 con 0x561aef0d64d0 2024-05-17T04:48:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.678+0000 7f8eb77fe640 1 -- 172.21.15.50:0/2949909180 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc030110 con 0x561aef0d64d0 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 -- 172.21.15.50:0/2949909180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 msgr2=0x561aef4ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x561aef4b1d40 tx=0x7f8ebc016000 comp rx=0 tx=0).stop 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 -- 172.21.15.50:0/2949909180 shutdown_connections 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4ab460 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 --2- 172.21.15.50:0/2949909180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef49eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 -- 172.21.15.50:0/2949909180 >> 172.21.15.50:0/2949909180 conn(0x561aef0eb300 msgr2=0x561aef4996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 -- 172.21.15.50:0/2949909180 shutdown_connections 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.679+0000 7f8edb026740 1 -- 172.21.15.50:0/2949909180 wait complete. 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8edb026740 1 -- start start 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8edb026740 1 Processor -- start 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8edb026740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef4aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8edb026740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef4aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8ed1e2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef4aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40124/0 (socket says 172.21.15.50:40124) 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8ed162a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.680+0000 7f8ed162a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57260/0 (socket says 172.21.15.50:57260) 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed1e2b640 1 -- 172.21.15.50:0/2289016639 learned_addr learned my addr 172.21.15.50:0/2289016639 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b4fa0 con 0x561aef0d64d0 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b6410 con 0x561aef0ce2f0 2024-05-17T04:48:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1b5670 con 0x561aef0f71b0 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8edb026740 10 client.0 _ll_register_callbacks cb 0x7f8ebc009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 -- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 msgr2=0x561aef4aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef4aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 -- 172.21.15.50:0/2289016639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 msgr2=0x561aef4aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aef517aa0 con 0x561aef0f71b0 2024-05-17T04:48:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed162a640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:48:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8ed262c640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4100ab0 tx=0x7f8ec402fe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cd42f38e9b4c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:48:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.681+0000 7f8eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:48:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec402c040 con 0x561aef0f71b0 2024-05-17T04:48:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8ec402fb10 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ebc0054b0 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ec40ca580 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8edb026740 10 client.26023 fetch_fsmap learned FSMap version 39 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.682+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561aef518e10 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ec40ce930 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec404d030 con 0x561aef0f71b0 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 10 client.26023 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 10 client.26023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:48:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 20 client.26023 populate_metadata read hostname 'smithi050' 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef08b1c0 0x561aef51b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 4 client.26023 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8ebc0054b0 con 0x561aef08b1c0 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.683+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef08b1c0 0x561aef51b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.684+0000 7f8ed1e2b640 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef08b1c0 0x561aef51b7b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22717ef888c883f6 server_cookie=e581a453ae9374f in_seq=0 out_seq=0 2024-05-17T04:48:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.684+0000 7f8eb57fa640 10 client.26023 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8eb57fa640 1 -- 172.21.15.50:0/2289016639 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f8ebc0054b0 con 0x561aef08b1c0 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8eb57fa640 10 client.26023 handle_command_reply: tid=0 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 client.26023 shutdown 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 msgr2=0x561aef4ab0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4100ab0 tx=0x7f8ec402fe30 comp rx=0 tx=0).stop 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 shutdown_connections 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561aef0f71b0 0x561aef4ab0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561aef08b1c0 0x561aef51b7b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561aef0d64d0 0x561aef4aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 --2- 172.21.15.50:0/2289016639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561aef0ce2f0 0x561aef4aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8eb57fa640 0 client.26023 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 >> 172.21.15.50:0/2289016639 conn(0x561aef0eb300 msgr2=0x561aef49db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 shutdown_connections 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.726+0000 7f8edb026740 1 -- 172.21.15.50:0/2289016639 wait complete. 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8edb026740 20 client.26023 trim_cache size 0 max 16384 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:48:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 msgr2=0x7f8eb80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 0x7f8eb80b4cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x561aef49b960 tx=0x7f8ebc004730 comp rx=0 tx=0).stop 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 msgr2=0x7f8ecc1ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc1ab680 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402ae70 tx=0x7f8ec4003ba0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 shutdown_connections 2024-05-17T04:48:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f8eb80b27e0 0x7f8eb80b4cf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ecc110590 0x7f8ecc1ab680 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ecc10dc30 0x7f8ecc1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 --2- 172.21.15.50:0/3889302194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ecc109f20 0x7f8ecc1a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 >> 172.21.15.50:0/3889302194 conn(0x7f8ecc0701c0 msgr2=0x7f8ecc1109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 shutdown_connections 2024-05-17T04:48:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:48:56.727+0000 7f8ed408f640 1 -- 172.21.15.50:0/3889302194 wait complete. 2024-05-17T04:48:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 600.0 s, 119.0 tps, lat 825.442 ms stddev 224.430 2024-05-17T04:48:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:58 smithi105 ceph-mon[34795]: pgmap v818: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 402 op/s 2024-05-17T04:48:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:58 smithi110 ceph-mon[35497]: pgmap v818: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 402 op/s 2024-05-17T04:48:58.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:58 smithi050 ceph-mon[31739]: pgmap v818: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 402 op/s 2024-05-17T04:48:59.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:48:59 smithi050 ceph-mon[31739]: pgmap v819: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-05-17T04:48:59.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:48:59 smithi105 ceph-mon[34795]: pgmap v819: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-05-17T04:48:59.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:48:59 smithi110 ceph-mon[35497]: pgmap v819: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-05-17T04:49:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:02 smithi105 ceph-mon[34795]: pgmap v820: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-05-17T04:49:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:02 smithi110 ceph-mon[35497]: pgmap v820: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-05-17T04:49:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:02 smithi050 ceph-mon[31739]: pgmap v820: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-05-17T04:49:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 605.0 s, 139.8 tps, lat 764.369 ms stddev 210.994 2024-05-17T04:49:04.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:04 smithi105 ceph-mon[34795]: pgmap v821: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-05-17T04:49:04.368 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:04 smithi110 ceph-mon[35497]: pgmap v821: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-05-17T04:49:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:04 smithi050 ceph-mon[31739]: pgmap v821: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-05-17T04:49:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:06 smithi105 ceph-mon[34795]: pgmap v822: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-05-17T04:49:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:06 smithi110 ceph-mon[35497]: pgmap v822: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-05-17T04:49:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:06 smithi050 ceph-mon[31739]: pgmap v822: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 2.3 MiB/s wr, 361 op/s 2024-05-17T04:49:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 610.0 s, 121.0 tps, lat 763.478 ms stddev 234.656 2024-05-17T04:49:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:08 smithi105 ceph-mon[34795]: pgmap v823: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:49:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:08 smithi110 ceph-mon[35497]: pgmap v823: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:49:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:08 smithi050 ceph-mon[31739]: pgmap v823: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 2.3 MiB/s wr, 379 op/s 2024-05-17T04:49:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 -- 172.21.15.50:0/859539441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8102080 msgr2=0x7fe6a8102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/859539441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8102080 0x7fe6a8102490 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009900 tx=0x7fe6a402a410 comp rx=0 tx=0).stop 2024-05-17T04:49:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 -- 172.21.15.50:0/859539441 shutdown_connections 2024-05-17T04:49:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/859539441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a8103af0 0x7fe6a810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/859539441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a81031a0 0x7fe6a81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/859539441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8102080 0x7fe6a8102490 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 -- 172.21.15.50:0/859539441 >> 172.21.15.50:0/859539441 conn(0x7fe6a80fd880 msgr2=0x7fe6a80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 -- 172.21.15.50:0/859539441 shutdown_connections 2024-05-17T04:49:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.862+0000 7fe6afbe7640 1 -- 172.21.15.50:0/859539441 wait complete. 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.863+0000 7fe6afbe7640 1 Processor -- start 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.863+0000 7fe6afbe7640 1 -- start start 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a8102080 0x7fe6a81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8103af0 0x7fe6a819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810dab0 con 0x7fe6a8103af0 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810d930 con 0x7fe6a81031a0 2024-05-17T04:49:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6afbe7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a810dc30 con 0x7fe6a8102080 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ae184640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8103af0 0x7fe6a819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:41934/0 (socket says 172.21.15.50:41934) 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 -- 172.21.15.50:0/1503432972 learned_addr learned my addr 172.21.15.50:0/1503432972 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad983640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a8102080 0x7fe6a81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 -- 172.21.15.50:0/1503432972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a8102080 msgr2=0x7fe6a81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a8102080 0x7fe6a81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 -- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8103af0 msgr2=0x7fe6a819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8103af0 0x7fe6a819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.864+0000 7fe6ad182640 1 -- 172.21.15.50:0/1503432972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a81a0210 con 0x7fe6a81031a0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe6ad182640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe69800eb00 tx=0x7fe69800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe69effd640 1 -- 172.21.15.50:0/1503432972 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69800bc30 con 0x7fe6a81031a0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a806a5c0 con 0x7fe6a81031a0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a806ab10 con 0x7fe6a81031a0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe69effd640 1 -- 172.21.15.50:0/1503432972 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe69800bdd0 con 0x7fe6a81031a0 2024-05-17T04:49:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.865+0000 7fe69effd640 1 -- 172.21.15.50:0/1503432972 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004e60 con 0x7fe6a81031a0 2024-05-17T04:49:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe69effd640 1 -- 172.21.15.50:0/1503432972 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe698017510 con 0x7fe6a81031a0 2024-05-17T04:49:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 Processor -- start 2024-05-17T04:49:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 -- start start 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f395dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f3964c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a8f38cfcc0 con 0x55a8f35ba1b0 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8f38bf860 con 0x55a8f35912f0 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.868+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a8f38c38a0 con 0x55a8f35994d0 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe69effd640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 0x7fe6940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f395dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36516/0 (socket says 172.21.15.50:36516) 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 -- 172.21.15.50:0/2601544289 learned_addr learned my addr 172.21.15.50:0/2601544289 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ad182640 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f3964c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 0x7fe6940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe69effd640 1 -- 172.21.15.50:0/1503432972 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe6980e3d40 con 0x7fe6a81031a0 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 -- 172.21.15.50:0/2601544289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 msgr2=0x55a8f3964c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f3964c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 -- 172.21.15.50:0/2601544289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 msgr2=0x55a8f395dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f395dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 -- 172.21.15.50:0/2601544289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f3887d50 con 0x55a8f35ba1b0 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ad983640 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f395dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.869+0000 7fe6ae184640 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005c40 tx=0x7fe6a000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51ba2525e109ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.870+0000 7fe693fff640 1 -- 172.21.15.50:0/2601544289 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0022020 con 0x55a8f35ba1b0 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.870+0000 7fe693fff640 1 -- 172.21.15.50:0/2601544289 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a0013170 con 0x55a8f35ba1b0 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.870+0000 7fe6ae184640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 0x7fe6940b4d70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0023000 tx=0x7fe6a0021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.870+0000 7fe693fff640 1 -- 172.21.15.50:0/2601544289 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0010690 con 0x55a8f35ba1b0 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2601544289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 msgr2=0x55a8f395d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005c40 tx=0x7fe6a000f3c0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2601544289 shutdown_connections 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f3964c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f395dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2601544289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f395d770 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2601544289 >> 172.21.15.50:0/2601544289 conn(0x55a8f35ae300 msgr2=0x55a8f39591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2601544289 shutdown_connections 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2601544289 wait complete. 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 -- start start 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.871+0000 7fe6b6b78740 1 Processor -- start 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f39675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a8f37fbab0 con 0x55a8f35ba1b0 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8f387c4d0 con 0x55a8f35912f0 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3965690 con 0x55a8f35994d0 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:41946/0 (socket says 172.21.15.50:41946) 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 -- 172.21.15.50:0/3017765102 learned_addr learned my addr 172.21.15.50:0/3017765102 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad182640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f39675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ae184640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6b6b78740 10 client.0 _ll_register_callbacks cb 0x55a8f38ba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 -- 172.21.15.50:0/3017765102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 msgr2=0x55a8f39675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f39675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 -- 172.21.15.50:0/3017765102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 msgr2=0x55a8f396e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 -- 172.21.15.50:0/3017765102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f387d340 con 0x55a8f35912f0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ae184640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6ad983640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619a37dca12ba799 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.872+0000 7fe6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4035040 con 0x55a8f35912f0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a0013170 con 0x55a8f35912f0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe6a4003a00 con 0x55a8f35912f0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6a404e050 con 0x55a8f35912f0 2024-05-17T04:49:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6b6b78740 10 client.26060 fetch_fsmap learned FSMap version 39 2024-05-17T04:49:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8f3625790 con 0x55a8f35912f0 2024-05-17T04:49:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.873+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a0013170 con 0x55a8f35912f0 2024-05-17T04:49:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe6a403c440 con 0x55a8f35912f0 2024-05-17T04:49:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 10 client.26060 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 20 client.26060 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f3578d80 0x55a8f3974860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 4 client.26060 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8f38d1c10 con 0x55a8f3578d80 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.874+0000 7fe6ad182640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f3578d80 0x55a8f3974860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6917fa640 10 client.26060 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6ad182640 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f3578d80 0x55a8f3974860 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4263fec25d28e6 server_cookie=3cda1ff035660ebd in_seq=0 out_seq=0 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6917fa640 1 -- 172.21.15.50:0/3017765102 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8f38d1c10 con 0x55a8f3578d80 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6917fa640 10 client.26060 handle_command_reply: tid=0 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6b6b78740 1 client.26060 shutdown 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 msgr2=0x55a8f3967060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.875+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40043b0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 shutdown_connections 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f3578d80 0x55a8f3974860 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35994d0 0x55a8f39675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/3017765102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35912f0 0x55a8f3967060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 >> 172.21.15.50:0/3017765102 conn(0x55a8f35ae300 msgr2=0x55a8f39581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6917fa640 0 client.26060 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 shutdown_connections 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 1 -- 172.21.15.50:0/3017765102 wait complete. 2024-05-17T04:49:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.876+0000 7fe6b6b78740 20 client.26060 trim_cache size 0 max 16384 2024-05-17T04:49:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.897+0000 7fe6b6b78740 1 Processor -- start 2024-05-17T04:49:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.897+0000 7fe6b6b78740 1 -- start start 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f396e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81ac590 con 0x55a8f35912f0 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81a9e60 con 0x55a8f35994d0 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81a7cb0 con 0x55a8f35ba1b0 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36526/0 (socket says 172.21.15.50:36526) 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 -- 172.21.15.50:0/393453417 learned_addr learned my addr 172.21.15.50:0/393453417 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad182640 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f396e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ae184640 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 -- 172.21.15.50:0/393453417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 msgr2=0x55a8f396e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 -- 172.21.15.50:0/393453417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 msgr2=0x55a8f396e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f396e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.898+0000 7fe6ad983640 1 -- 172.21.15.50:0/393453417 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f369e620 con 0x55a8f35912f0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.899+0000 7fe6ad983640 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009850 tx=0x7fe6a4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca88278037c9184 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.899+0000 7fe6937fe640 1 -- 172.21.15.50:0/393453417 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a404b070 con 0x55a8f35912f0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.899+0000 7fe6937fe640 1 -- 172.21.15.50:0/393453417 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a4005850 con 0x55a8f35912f0 2024-05-17T04:49:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.899+0000 7fe6937fe640 1 -- 172.21.15.50:0/393453417 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4035070 con 0x55a8f35912f0 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 -- 172.21.15.50:0/393453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 msgr2=0x55a8f3957ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009850 tx=0x7fe6a4005d50 comp rx=0 tx=0).stop 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 -- 172.21.15.50:0/393453417 shutdown_connections 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f396e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/393453417 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3957ef0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 -- 172.21.15.50:0/393453417 >> 172.21.15.50:0/393453417 conn(0x55a8f35ae300 msgr2=0x55a8f39577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 -- 172.21.15.50:0/393453417 shutdown_connections 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.900+0000 7fe6b6b78740 1 -- 172.21.15.50:0/393453417 wait complete. 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 -- start start 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 Processor -- start 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f3969540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81a8320 con 0x55a8f35912f0 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81a9790 con 0x55a8f35994d0 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81a89f0 con 0x55a8f35ba1b0 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6b6b78740 10 client.0 _ll_register_callbacks cb 0x7fe6a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.901+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36528/0 (socket says 172.21.15.50:36528) 2024-05-17T04:49:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 -- 172.21.15.50:0/2979396987 learned_addr learned my addr 172.21.15.50:0/2979396987 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad182640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f3969540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ae184640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 -- 172.21.15.50:0/2979396987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 msgr2=0x55a8f396e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 -- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 msgr2=0x55a8f3969540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f3969540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 -- 172.21.15.50:0/2979396987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f396ea80 con 0x55a8f35912f0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad182640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f3969540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6ad983640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4004150 tx=0x7fe6a4003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e03113d1eb78fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4035030 con 0x55a8f35912f0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe6a403c440 con 0x55a8f35912f0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.902+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a4005850 con 0x55a8f35912f0 2024-05-17T04:49:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6a404b040 con 0x55a8f35912f0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 10 client.16311 fetch_fsmap learned FSMap version 39 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a8f39dba00 con 0x55a8f35912f0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6a4005850 con 0x55a8f35912f0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe6a4053440 con 0x55a8f35912f0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 10 client.16311 fetch_fsmap finished waiting for FSMap version 39 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 20 client.16311 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f354e1c0 0x55a8f39de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 4 client.16311 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.903+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8f38cb3f0 con 0x55a8f354e1c0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.904+0000 7fe6ad182640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f354e1c0 0x55a8f39de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.904+0000 7fe6917fa640 10 client.16311 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.904+0000 7fe6ad182640 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f354e1c0 0x55a8f39de370 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0df5161675d702 server_cookie=35f8bfaf63fef1e4 in_seq=0 out_seq=0 2024-05-17T04:49:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6917fa640 1 -- 172.21.15.50:0/2979396987 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a8f38cb3f0 con 0x55a8f354e1c0 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6917fa640 10 client.16311 handle_command_reply: tid=0 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 client.16311 shutdown 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 msgr2=0x55a8f3969000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4004150 tx=0x7fe6a4003ab0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 shutdown_connections 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8f35ba1b0 0x55a8f396e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f35994d0 0x55a8f3969540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a8f354e1c0 0x55a8f39de370 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 --2- 172.21.15.50:0/2979396987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a8f35912f0 0x55a8f3969000 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6917fa640 0 client.16311 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.939+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 >> 172.21.15.50:0/2979396987 conn(0x55a8f35ae300 msgr2=0x55a8f395bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 shutdown_connections 2024-05-17T04:49:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6b6b78740 1 -- 172.21.15.50:0/2979396987 wait complete. 2024-05-17T04:49:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6b6b78740 20 client.16311 trim_cache size 0 max 16384 2024-05-17T04:49:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:49:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:49:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 msgr2=0x7fe6940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 0x7fe6940b4d70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0023000 tx=0x7fe6a0021000 comp rx=0 tx=0).stop 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 msgr2=0x7fe6a8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe69800eb00 tx=0x7fe69800ef00 comp rx=0 tx=0).stop 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 shutdown_connections 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe6940b2860 0x7fe6940b4d70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe6a8103af0 0x7fe6a819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a81031a0 0x7fe6a8198c10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 --2- 172.21.15.50:0/1503432972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6a8102080 0x7fe6a81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 >> 172.21.15.50:0/1503432972 conn(0x7fe6a80fd880 msgr2=0x7fe6a81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.940+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 shutdown_connections 2024-05-17T04:49:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:09.941+0000 7fe6afbe7640 1 -- 172.21.15.50:0/1503432972 wait complete. 2024-05-17T04:49:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:10 smithi105 ceph-mon[34795]: pgmap v824: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:49:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:10 smithi110 ceph-mon[35497]: pgmap v824: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:49:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:10 smithi050 ceph-mon[31739]: pgmap v824: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.4 MiB/s wr, 355 op/s 2024-05-17T04:49:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:12 smithi105 ceph-mon[34795]: pgmap v825: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.1 MiB/s wr, 340 op/s 2024-05-17T04:49:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:12 smithi110 ceph-mon[35497]: pgmap v825: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.1 MiB/s wr, 340 op/s 2024-05-17T04:49:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:12 smithi050 ceph-mon[31739]: pgmap v825: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.1 MiB/s wr, 340 op/s 2024-05-17T04:49:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 615.0 s, 133.6 tps, lat 773.826 ms stddev 220.021 2024-05-17T04:49:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:14 smithi105 ceph-mon[34795]: pgmap v826: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-05-17T04:49:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:14 smithi110 ceph-mon[35497]: pgmap v826: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-05-17T04:49:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:14 smithi050 ceph-mon[31739]: pgmap v826: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 2.4 MiB/s wr, 389 op/s 2024-05-17T04:49:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:16 smithi105 ceph-mon[34795]: pgmap v827: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-05-17T04:49:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:16 smithi110 ceph-mon[35497]: pgmap v827: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-05-17T04:49:16.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:16 smithi050 ceph-mon[31739]: pgmap v827: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.4 MiB/s wr, 370 op/s 2024-05-17T04:49:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 620.0 s, 137.2 tps, lat 709.059 ms stddev 240.118 2024-05-17T04:49:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:18 smithi105 ceph-mon[34795]: pgmap v828: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-05-17T04:49:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:18 smithi110 ceph-mon[35497]: pgmap v828: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-05-17T04:49:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:18 smithi050 ceph-mon[31739]: pgmap v828: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-05-17T04:49:20.378 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:20 smithi050 ceph-mon[31739]: pgmap v829: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.4 MiB/s wr, 368 op/s 2024-05-17T04:49:20.378 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:20 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:20 smithi105 ceph-mon[34795]: pgmap v829: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.4 MiB/s wr, 368 op/s 2024-05-17T04:49:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:20 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:20 smithi110 ceph-mon[35497]: pgmap v829: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 2.4 MiB/s wr, 368 op/s 2024-05-17T04:49:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:20 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:22 smithi105 ceph-mon[34795]: pgmap v830: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:22 smithi110 ceph-mon[35497]: pgmap v830: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:22 smithi050 ceph-mon[31739]: pgmap v830: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 625.0 s, 107.2 tps, lat 937.580 ms stddev 214.335 2024-05-17T04:49:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 -- 172.21.15.50:0/2004976558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb500103c70 msgr2=0x7fb500106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 --2- 172.21.15.50:0/2004976558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb500103c70 0x7fb500106090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009900 tx=0x7fb4fc02a410 comp rx=0 tx=0).stop 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 -- 172.21.15.50:0/2004976558 shutdown_connections 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 --2- 172.21.15.50:0/2004976558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 0x7fb50010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 --2- 172.21.15.50:0/2004976558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5001065d0 0x7fb5001089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 --2- 172.21.15.50:0/2004976558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb500103c70 0x7fb500106090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 -- 172.21.15.50:0/2004976558 >> 172.21.15.50:0/2004976558 conn(0x7fb5000fd880 msgr2=0x7fb5000ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 -- 172.21.15.50:0/2004976558 shutdown_connections 2024-05-17T04:49:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.092+0000 7fb508119640 1 -- 172.21.15.50:0/2004976558 wait complete. 2024-05-17T04:49:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.093+0000 7fb508119640 1 Processor -- start 2024-05-17T04:49:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.093+0000 7fb508119640 1 -- start start 2024-05-17T04:49:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.093+0000 7fb508119640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb500103c70 0x7fb5001997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb508119640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb508119640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 0x7fb5001a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb508119640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010eb50 con 0x7fb5001065d0 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb508119640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010e9d0 con 0x7fb500108f30 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb508119640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010ecd0 con 0x7fb500103c70 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33874/0 (socket says 172.21.15.50:33874) 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 -- 172.21.15.50:0/1218582706 learned_addr learned my addr 172.21.15.50:0/1218582706 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb505eb5640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb500103c70 0x7fb5001997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5066b6640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 0x7fb5001a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 -- 172.21.15.50:0/1218582706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb500103c70 msgr2=0x7fb5001997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb500103c70 0x7fb5001997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 -- 172.21.15.50:0/1218582706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 msgr2=0x7fb5001a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 0x7fb5001a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 -- 172.21.15.50:0/1218582706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5001a1330 con 0x7fb5001065d0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.094+0000 7fb5056b4640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ea60 tx=0x7fb4f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.095+0000 7fb4f6ffd640 1 -- 172.21.15.50:0/1218582706 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000cde0 con 0x7fb5001065d0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.095+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5001a1620 con 0x7fb5001065d0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.095+0000 7fb4f6ffd640 1 -- 172.21.15.50:0/1218582706 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4f00035f0 con 0x7fb5001065d0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.095+0000 7fb4f6ffd640 1 -- 172.21.15.50:0/1218582706 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0010690 con 0x7fb5001065d0 2024-05-17T04:49:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.095+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb500101ca0 con 0x7fb5001065d0 2024-05-17T04:49:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.099+0000 7fb4f6ffd640 1 -- 172.21.15.50:0/1218582706 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb4f0010830 con 0x7fb5001065d0 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.100+0000 7fb4f6ffd640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 0x7fb4ec0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.100+0000 7fb505eb5640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 0x7fb4ec0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.100+0000 7fb4f6ffd640 1 -- 172.21.15.50:0/1218582706 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb4f00e2850 con 0x7fb5001065d0 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.100+0000 7fb50efd7740 1 Processor -- start 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 -- start start 2024-05-17T04:49:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbb111b0 0x561ddbeb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbeb4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561ddbe26cc0 con 0x561ddbb111b0 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561ddbe16860 con 0x561ddbaf04d0 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb50efd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561ddbe1a8a0 con 0x561ddbae82f0 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbeb4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbeb4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:53348/0 (socket says 172.21.15.50:53348) 2024-05-17T04:49:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb5056b4640 1 -- 172.21.15.50:0/3221972578 learned_addr learned my addr 172.21.15.50:0/3221972578 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb505eb5640 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbb111b0 0x561ddbeb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.101+0000 7fb505eb5640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 0x7fb4ec0b4cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005ab0 tx=0x7fb4fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 -- 172.21.15.50:0/3221972578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 msgr2=0x561ddbeb4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbeb4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 -- 172.21.15.50:0/3221972578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbb111b0 msgr2=0x561ddbeb4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbb111b0 0x561ddbeb4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 -- 172.21.15.50:0/3221972578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ddbdded50 con 0x561ddbaf04d0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebbc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009f00 tx=0x7fb4f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=643d3e36d08b8e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb4ebfff640 1 -- 172.21.15.50:0/3221972578 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8004d20 con 0x561ddbaf04d0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb4ebfff640 1 -- 172.21.15.50:0/3221972578 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4f8004030 con 0x561ddbaf04d0 2024-05-17T04:49:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.102+0000 7fb4ebfff640 1 -- 172.21.15.50:0/3221972578 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8005280 con 0x561ddbaf04d0 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.103+0000 7fb50efd7740 1 -- 172.21.15.50:0/3221972578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 msgr2=0x561ddbebbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.103+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebbc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009f00 tx=0x7fb4f8011810 comp rx=0 tx=0).stop 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 -- 172.21.15.50:0/3221972578 shutdown_connections 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebbc10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbeb4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3221972578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbb111b0 0x561ddbeb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 -- 172.21.15.50:0/3221972578 >> 172.21.15.50:0/3221972578 conn(0x561ddbb05300 msgr2=0x561ddbeb01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 -- 172.21.15.50:0/3221972578 shutdown_connections 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.104+0000 7fb50efd7740 1 -- 172.21.15.50:0/3221972578 wait complete. 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.105+0000 7fb50efd7740 1 -- start start 2024-05-17T04:49:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.105+0000 7fb50efd7740 1 Processor -- start 2024-05-17T04:49:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.105+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 0x561ddbec5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33888/0 (socket says 172.21.15.50:33888) 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb505eb5640 1 -- 172.21.15.50:0/3990903473 learned_addr learned my addr 172.21.15.50:0/3990903473 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb5056b4640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561ddbd52ab0 con 0x561ddbae82f0 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 0x561ddbec5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561ddbdd34d0 con 0x561ddbaf04d0 2024-05-17T04:49:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561ddbebc690 con 0x561ddbb111b0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.106+0000 7fb50efd7740 10 client.0 _ll_register_callbacks cb 0x561ddbe11950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 -- 172.21.15.50:0/3990903473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 msgr2=0x561ddbec5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 0x561ddbec5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 -- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 msgr2=0x561ddbebe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ddbdd4340 con 0x561ddbae82f0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb5066b6640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 0x561ddbec5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb5056b4640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb505eb5640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb50006af80 tx=0x7fb4fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4108f89363cc5ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc035050 con 0x561ddbae82f0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f8004030 con 0x561ddbae82f0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb4fc0035f0 con 0x561ddbae82f0 2024-05-17T04:49:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc05b050 con 0x561ddbae82f0 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.107+0000 7fb50efd7740 10 client.16323 fetch_fsmap learned FSMap version 40 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ddbb7c790 con 0x561ddbae82f0 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4f8004030 con 0x561ddbae82f0 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fb4fc0527f0 con 0x561ddbae82f0 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 10 client.16323 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 20 client.16323 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbacfd80 0x561ddbecb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 4 client.16323 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:49:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ddbe28c10 con 0x561ddbacfd80 2024-05-17T04:49:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.108+0000 7fb5056b4640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbacfd80 0x561ddbecb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.109+0000 7fb4e97fa640 10 client.16323 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.109+0000 7fb5056b4640 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbacfd80 0x561ddbecb8f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56ca6d64acb68ea server_cookie=a8e21f8b87843b3f in_seq=0 out_seq=0 2024-05-17T04:49:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb4e97fa640 1 -- 172.21.15.50:0/3990903473 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ddbe28c10 con 0x561ddbacfd80 2024-05-17T04:49:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb4e97fa640 10 client.16323 handle_command_reply: tid=0 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 client.16323 shutdown 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 msgr2=0x561ddbebe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb50006af80 tx=0x7fb4fc009260 comp rx=0 tx=0).stop 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 shutdown_connections 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbb111b0 0x561ddbec5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbebe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbacfd80 0x561ddbecb8f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb4e97fa640 0 client.16323 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 --2- 172.21.15.50:0/3990903473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbebe030 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 >> 172.21.15.50:0/3990903473 conn(0x561ddbb05300 msgr2=0x561ddbeaf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 shutdown_connections 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 1 -- 172.21.15.50:0/3990903473 wait complete. 2024-05-17T04:49:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.110+0000 7fb50efd7740 20 client.16323 trim_cache size 0 max 16384 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 Processor -- start 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 -- start start 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbeb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbaf04d0 0x561ddbeb7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001b0a60 con 0x561ddbae82f0 2024-05-17T04:49:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001ae330 con 0x561ddbacfd80 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.131+0000 7fb50efd7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001ac180 con 0x561ddbaf04d0 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbeb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52024/0 (socket says 172.21.15.50:52024) 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb5056b4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbeb78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33892/0 (socket says 172.21.15.50:33892) 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb5066b6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbaf04d0 0x561ddbeb7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 -- 172.21.15.50:0/2788778373 learned_addr learned my addr 172.21.15.50:0/2788778373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 -- 172.21.15.50:0/2788778373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbaf04d0 msgr2=0x561ddbeb7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbaf04d0 0x561ddbeb7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 -- 172.21.15.50:0/2788778373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 msgr2=0x561ddbeb78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbeb78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 -- 172.21.15.50:0/2788778373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ddbbf5620 con 0x561ddbacfd80 2024-05-17T04:49:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.132+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc052db0 tx=0x7fb4fc02ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d2c5948c0adf0b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.133+0000 7fb4eb7fe640 1 -- 172.21.15.50:0/2788778373 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc064070 con 0x561ddbacfd80 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.133+0000 7fb4eb7fe640 1 -- 172.21.15.50:0/2788778373 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4fc04d070 con 0x561ddbacfd80 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.133+0000 7fb4eb7fe640 1 -- 172.21.15.50:0/2788778373 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc035040 con 0x561ddbacfd80 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- 172.21.15.50:0/2788778373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 msgr2=0x561ddbeaeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc052db0 tx=0x7fb4fc02ab10 comp rx=0 tx=0).stop 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- 172.21.15.50:0/2788778373 shutdown_connections 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbaf04d0 0x561ddbeb7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbae82f0 0x561ddbeb78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2788778373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbacfd80 0x561ddbeaeef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- 172.21.15.50:0/2788778373 >> 172.21.15.50:0/2788778373 conn(0x561ddbb05300 msgr2=0x561ddbeae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- 172.21.15.50:0/2788778373 shutdown_connections 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- 172.21.15.50:0/2788778373 wait complete. 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.134+0000 7fb50efd7740 1 -- start start 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 Processor -- start 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbec2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb505eb5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33904/0 (socket says 172.21.15.50:33904) 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb505eb5640 1 -- 172.21.15.50:0/2599188884 learned_addr learned my addr 172.21.15.50:0/2599188884 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbec37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001ac7f0 con 0x561ddbacfd80 2024-05-17T04:49:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001adc60 con 0x561ddbaf04d0 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001acec0 con 0x561ddbae82f0 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.135+0000 7fb50efd7740 10 client.0 _ll_register_callbacks cb 0x7fb4fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb5056b4640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbec2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb5066b6640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbec37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 -- 172.21.15.50:0/2599188884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 msgr2=0x561ddbec2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbec2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 -- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 msgr2=0x561ddbec37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbec37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ddbec3da0 con 0x561ddbacfd80 2024-05-17T04:49:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb505eb5640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc052d30 tx=0x7fb4fc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a8eadde7073def server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb4e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:49:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.136+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc064040 con 0x561ddbacfd80 2024-05-17T04:49:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb4fc052670 con 0x561ddbacfd80 2024-05-17T04:49:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc04d070 con 0x561ddbacfd80 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc035040 con 0x561ddbacfd80 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb50efd7740 10 client.16329 fetch_fsmap learned FSMap version 40 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ddbf3c470 con 0x561ddbacfd80 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.137+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4fc04d070 con 0x561ddbacfd80 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fb4fc069600 con 0x561ddbacfd80 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 10 client.16329 fetch_fsmap finished waiting for FSMap version 40 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 20 client.16329 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbb111b0 0x561ddbf3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 4 client.16329 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:49:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ddbe223f0 con 0x561ddbb111b0 2024-05-17T04:49:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.138+0000 7fb5056b4640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbb111b0 0x561ddbf3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.139+0000 7fb4e97fa640 10 client.16329 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.139+0000 7fb5056b4640 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbb111b0 0x561ddbf3ec40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cbab4cb9895c53 server_cookie=f41199a00eb714bd in_seq=0 out_seq=0 2024-05-17T04:49:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.180+0000 7fb4e97fa640 1 -- 172.21.15.50:0/2599188884 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561ddbe223f0 con 0x561ddbb111b0 2024-05-17T04:49:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.180+0000 7fb4e97fa640 10 client.16329 handle_command_reply: tid=0 2024-05-17T04:49:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.180+0000 7fb50efd7740 1 client.16329 shutdown 2024-05-17T04:49:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.180+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 msgr2=0x561ddbec2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.180+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc052d30 tx=0x7fb4fc0045a0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 shutdown_connections 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ddbaf04d0 0x561ddbec37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ddbae82f0 0x561ddbec2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561ddbb111b0 0x561ddbf3ec40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 --2- 172.21.15.50:0/2599188884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ddbacfd80 0x561ddbec2340 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb4e97fa640 0 client.16329 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 >> 172.21.15.50:0/2599188884 conn(0x561ddbb05300 msgr2=0x561ddbeae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 shutdown_connections 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 1 -- 172.21.15.50:0/2599188884 wait complete. 2024-05-17T04:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.181+0000 7fb50efd7740 20 client.16329 trim_cache size 0 max 16384 2024-05-17T04:49:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:49:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:49:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:49:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 msgr2=0x7fb4ec0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 0x7fb4ec0b4cf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005ab0 tx=0x7fb4fc000bc0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 msgr2=0x7fb500199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000ea60 tx=0x7fb4f000ef20 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 shutdown_connections 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb500108f30 0x7fb5001a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb4ec0b27e0 0x7fb4ec0b4cf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5001065d0 0x7fb500199d30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 --2- 172.21.15.50:0/1218582706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb500103c70 0x7fb5001997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 >> 172.21.15.50:0/1218582706 conn(0x7fb5000fd880 msgr2=0x7fb500104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 shutdown_connections 2024-05-17T04:49:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:23.182+0000 7fb508119640 1 -- 172.21.15.50:0/1218582706 wait complete. 2024-05-17T04:49:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:24 smithi105 ceph-mon[34795]: pgmap v831: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 433 op/s 2024-05-17T04:49:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:24 smithi110 ceph-mon[35497]: pgmap v831: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 433 op/s 2024-05-17T04:49:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:24 smithi050 ceph-mon[31739]: pgmap v831: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.8 MiB/s wr, 433 op/s 2024-05-17T04:49:25.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:25 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:25.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:25 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:25.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:25 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:49:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:26 smithi105 ceph-mon[34795]: pgmap v832: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:49:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:26 smithi110 ceph-mon[35497]: pgmap v832: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:49:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:26 smithi050 ceph-mon[31739]: pgmap v832: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 2.5 MiB/s wr, 380 op/s 2024-05-17T04:49:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 630.0 s, 106.6 tps, lat 921.850 ms stddev 226.476 2024-05-17T04:49:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:28 smithi105 ceph-mon[34795]: pgmap v833: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 2.2 MiB/s wr, 370 op/s 2024-05-17T04:49:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:28 smithi110 ceph-mon[35497]: pgmap v833: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 2.2 MiB/s wr, 370 op/s 2024-05-17T04:49:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:28 smithi050 ceph-mon[31739]: pgmap v833: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 2.2 MiB/s wr, 370 op/s 2024-05-17T04:49:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:30 smithi105 ceph-mon[34795]: pgmap v834: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:49:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:30 smithi110 ceph-mon[35497]: pgmap v834: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:49:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:30 smithi050 ceph-mon[31739]: pgmap v834: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:49:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:32 smithi105 ceph-mon[34795]: pgmap v835: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:49:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:32 smithi110 ceph-mon[35497]: pgmap v835: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:49:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:32 smithi050 ceph-mon[31739]: pgmap v835: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 2.3 MiB/s wr, 353 op/s 2024-05-17T04:49:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 635.0 s, 84.4 tps, lat 1172.315 ms stddev 382.016 2024-05-17T04:49:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:34 smithi105 ceph-mon[34795]: pgmap v836: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-05-17T04:49:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:34 smithi110 ceph-mon[35497]: pgmap v836: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-05-17T04:49:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:34 smithi050 ceph-mon[31739]: pgmap v836: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.2 MiB/s wr, 376 op/s 2024-05-17T04:49:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 -- 172.21.15.50:0/3256568713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc1022e0 msgr2=0x7f1ecc1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/3256568713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc1026f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000d910 tx=0x7f1ec002a160 comp rx=0 tx=0).stop 2024-05-17T04:49:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 -- 172.21.15.50:0/3256568713 shutdown_connections 2024-05-17T04:49:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/3256568713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc102c30 0x7f1ecc10a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/3256568713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc1026f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/3256568713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 -- 172.21.15.50:0/3256568713 >> 172.21.15.50:0/3256568713 conn(0x7f1ecc0fd010 msgr2=0x7f1ecc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 -- 172.21.15.50:0/3256568713 shutdown_connections 2024-05-17T04:49:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.326+0000 7f1ed1a78640 1 -- 172.21.15.50:0/3256568713 wait complete. 2024-05-17T04:49:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 Processor -- start 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 -- start start 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 0x7f1ecc1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10c980 con 0x7f1ecc1022e0 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10c800 con 0x7f1ecc1035d0 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.327+0000 7f1ed1a78640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc10cb00 con 0x7f1ecc102c30 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54818/0 (socket says 172.21.15.50:54818) 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/1222703813 learned_addr learned my addr 172.21.15.50:0/1222703813 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 0x7f1ecc1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 -- 172.21.15.50:0/1222703813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 msgr2=0x7f1ecc1a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 -- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 msgr2=0x7f1ecc19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecaffd640 1 -- 172.21.15.50:0/1222703813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1a7960 con 0x7f1ecc102c30 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.328+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.329+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 0x7f1ecc1a0360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ec002a680 tx=0x7f1ec002ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.329+0000 7f1ec8ff9640 1 -- 172.21.15.50:0/1222703813 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec003f020 con 0x7f1ecc102c30 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.329+0000 7f1ec8ff9640 1 -- 172.21.15.50:0/1222703813 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec002acb0 con 0x7f1ecc102c30 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.330+0000 7f1ec8ff9640 1 -- 172.21.15.50:0/1222703813 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000c820 con 0x7f1ecc102c30 2024-05-17T04:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.330+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1a7bf0 con 0x7f1ecc102c30 2024-05-17T04:49:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.330+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1a8070 con 0x7f1ecc102c30 2024-05-17T04:49:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.333+0000 7f1ed8a0c740 1 Processor -- start 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 -- start start 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 0x55de56fa0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55de56f0bcc0 con 0x55de56bd54d0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55de56efb860 con 0x55de56bcd2f0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de56eff8a0 con 0x55de56bf61b0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.334+0000 7f1ec8ff9640 1 -- 172.21.15.50:0/1222703813 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ec0042020 con 0x7f1ecc102c30 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 0x55de56fa0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:49606/0 (socket says 172.21.15.50:49606) 2024-05-17T04:49:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:51678/0 (socket says 172.21.15.50:51678) 2024-05-17T04:49:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 -- 172.21.15.50:0/2301496407 learned_addr learned my addr 172.21.15.50:0/2301496407 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 -- 172.21.15.50:0/2301496407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 msgr2=0x55de56f99cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 -- 172.21.15.50:0/2301496407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 msgr2=0x55de56fa0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 0x55de56fa0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 -- 172.21.15.50:0/2301496407 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de56ec3d50 con 0x55de56bf61b0 2024-05-17T04:49:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 0x55de56fa0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.335+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005f40 tx=0x7f1ec400f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f3c0e08428bd03e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1eb1ffb640 1 -- 172.21.15.50:0/2301496407 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4018640 con 0x55de56bf61b0 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1eb1ffb640 1 -- 172.21.15.50:0/2301496407 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec4017870 con 0x55de56bf61b0 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1eb1ffb640 1 -- 172.21.15.50:0/2301496407 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4016950 con 0x55de56bf61b0 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1ec8ff9640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 0x7f1eb40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1ec8ff9640 1 -- 172.21.15.50:0/1222703813 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1ec0102080 con 0x7f1ecc102c30 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.336+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 0x7f1eb40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/2301496407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 msgr2=0x55de56f99770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4005f40 tx=0x7f1ec400f7f0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 0x7f1eb40b4d70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55de56f9ab70 tx=0x7f1ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/2301496407 shutdown_connections 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bd54d0 0x55de56fa0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56f99cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/2301496407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bf61b0 0x55de56f99770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/2301496407 >> 172.21.15.50:0/2301496407 conn(0x55de56bea300 msgr2=0x55de56f951a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/2301496407 shutdown_connections 2024-05-17T04:49:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.337+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/2301496407 wait complete. 2024-05-17T04:49:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.338+0000 7f1ed8a0c740 1 -- start start 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.338+0000 7f1ed8a0c740 1 Processor -- start 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.338+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.338+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56faa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55de56e37ab0 con 0x55de56bf61b0 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55de56eb84d0 con 0x55de56bcd2f0 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55de56fa1690 con 0x55de56bd54d0 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:51686/0 (socket says 172.21.15.50:51686) 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/3930513096 learned_addr learned my addr 172.21.15.50:0/3930513096 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ed8a0c740 10 client.0 _ll_register_callbacks cb 0x55de56ef6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56faa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/3930513096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 msgr2=0x55de56fa3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/3930513096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 msgr2=0x55de56faa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56faa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/3930513096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de56eb9340 con 0x55de56bcd2f0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011c40 tx=0x7f1ebc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be861ddbb40f0637 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.339+0000 7f1eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc010da0 con 0x55de56bcd2f0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1ebc028400 con 0x55de56bcd2f0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec4017870 con 0x55de56bcd2f0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc031050 con 0x55de56bcd2f0 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1ed8a0c740 10 client.26090 fetch_fsmap learned FSMap version 41 2024-05-17T04:49:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55de56c61790 con 0x55de56bcd2f0 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.340+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec4017870 con 0x55de56bcd2f0 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1ebc0206f0 con 0x55de56bcd2f0 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 10 client.26090 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 20 client.26090 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56bb4d80 0x55de56fb0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 4 client.26090 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de56f0dc10 con 0x55de56bb4d80 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.341+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56bb4d80 0x55de56fb0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.342+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56bb4d80 0x55de56fb0d50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f09183aa007938b server_cookie=d3e05d3f8e95dcad in_seq=0 out_seq=0 2024-05-17T04:49:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.342+0000 7f1eaaffd640 10 client.26090 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1eaaffd640 1 -- 172.21.15.50:0/3930513096 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de56f0dc10 con 0x55de56bb4d80 2024-05-17T04:49:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1eaaffd640 10 client.26090 handle_command_reply: tid=0 2024-05-17T04:49:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 client.26090 shutdown 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 msgr2=0x55de56fa30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011c40 tx=0x7f1ebc0046c0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 shutdown_connections 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56bb4d80 0x55de56fb0d50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56faa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.343+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/3930513096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56fa30c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.344+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 >> 172.21.15.50:0/3930513096 conn(0x55de56bea300 msgr2=0x55de56f98010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.344+0000 7f1eaaffd640 0 client.26090 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.344+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 shutdown_connections 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.344+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/3930513096 wait complete. 2024-05-17T04:49:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.344+0000 7f1ed8a0c740 20 client.26090 trim_cache size 0 max 16384 2024-05-17T04:49:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.369+0000 7f1ed8a0c740 1 Processor -- start 2024-05-17T04:49:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.369+0000 7f1ed8a0c740 1 -- start start 2024-05-17T04:49:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 0x55de56fa1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56f98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b4ec0 con 0x55de56bf61b0 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b2790 con 0x55de56bd54d0 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b05e0 con 0x55de56bcd2f0 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:49628/0 (socket says 172.21.15.50:49628) 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/440647351 learned_addr learned my addr 172.21.15.50:0/440647351 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 0x55de56fa1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56f98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecaffd640 1 -- 172.21.15.50:0/440647351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 msgr2=0x55de56f9dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.370+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecaffd640 1 -- 172.21.15.50:0/440647351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 msgr2=0x55de56f98260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56f98260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecaffd640 1 -- 172.21.15.50:0/440647351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de56cda620 con 0x55de56bd54d0 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56f98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.371+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 0x55de56fa1a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ff7c0 tx=0x7f1ec0005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ee404155c071a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.372+0000 7f1eb17fa640 1 -- 172.21.15.50:0/440647351 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00c8eb0 con 0x55de56bd54d0 2024-05-17T04:49:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.372+0000 7f1eb17fa640 1 -- 172.21.15.50:0/440647351 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec002e040 con 0x55de56bd54d0 2024-05-17T04:49:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.372+0000 7f1eb17fa640 1 -- 172.21.15.50:0/440647351 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00cd750 con 0x55de56bd54d0 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/440647351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 msgr2=0x55de56fa1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 0x55de56fa1a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ff7c0 tx=0x7f1ec0005750 comp rx=0 tx=0).stop 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/440647351 shutdown_connections 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56f98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bd54d0 0x55de56fa1a60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/440647351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bcd2f0 0x55de56f9dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/440647351 >> 172.21.15.50:0/440647351 conn(0x55de56bea300 msgr2=0x55de56f97b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.373+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/440647351 shutdown_connections 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.374+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/440647351 wait complete. 2024-05-17T04:49:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.374+0000 7f1ed8a0c740 1 -- start start 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.374+0000 7f1ed8a0c740 1 Processor -- start 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56fa9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b0c50 con 0x55de56bf61b0 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b20c0 con 0x55de56bcd2f0 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:51704/0 (socket says 172.21.15.50:51704) 2024-05-17T04:49:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/1822604348 learned_addr learned my addr 172.21.15.50:0/1822604348 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1b1320 con 0x55de56bd54d0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecbfff640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56fa9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ed8a0c740 10 client.0 _ll_register_callbacks cb 0x7f1ebc0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/1822604348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 msgr2=0x55de56fa9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/1822604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 msgr2=0x55de56fa9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.375+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56fa9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1ecb7fe640 1 -- 172.21.15.50:0/1822604348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de56fa9ab0 con 0x55de56bcd2f0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1ecb7fe640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097f0 tx=0x7f1ebc010dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbafb6992b3e2295 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc02e040 con 0x55de56bcd2f0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1ebc005040 con 0x55de56bcd2f0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec002e040 con 0x55de56bcd2f0 2024-05-17T04:49:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0208c0 con 0x55de56bcd2f0 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1ed8a0c740 10 client.26096 fetch_fsmap learned FSMap version 41 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.376+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55de57017ce0 con 0x55de56bcd2f0 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ec002e040 con 0x55de56bcd2f0 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1ebc004cd0 con 0x55de56bcd2f0 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 10 client.26096 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 10 client.26096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 20 client.26096 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56b8a1c0 0x55de5701a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 4 client.26096 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de56f073f0 con 0x55de56b8a1c0 2024-05-17T04:49:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.377+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56b8a1c0 0x55de5701a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.378+0000 7f1eaaffd640 10 client.26096 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.378+0000 7f1ecaffd640 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56b8a1c0 0x55de5701a710 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d63d70f15eb85f8 server_cookie=d3e8e3e06df12a40 in_seq=0 out_seq=0 2024-05-17T04:49:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1eaaffd640 1 -- 172.21.15.50:0/1822604348 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de56f073f0 con 0x55de56b8a1c0 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1eaaffd640 10 client.26096 handle_command_reply: tid=0 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 client.26096 shutdown 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 msgr2=0x55de56faa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097f0 tx=0x7f1ebc010dd0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 shutdown_connections 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55de56b8a1c0 0x55de5701a710 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de56bf61b0 0x55de56fa9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55de56bd54d0 0x55de56fa9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1eaaffd640 0 client.26096 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 --2- 172.21.15.50:0/1822604348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de56bcd2f0 0x55de56faa950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 >> 172.21.15.50:0/1822604348 conn(0x55de56bea300 msgr2=0x55de56f98510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 shutdown_connections 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 1 -- 172.21.15.50:0/1822604348 wait complete. 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.417+0000 7f1ed8a0c740 20 client.26096 trim_cache size 0 max 16384 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:49:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:49:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 msgr2=0x7f1eb40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 0x7f1eb40b4d70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55de56f9ab70 tx=0x7f1ebc005f90 comp rx=0 tx=0).stop 2024-05-17T04:49:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 msgr2=0x7f1ecc1a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 0x7f1ecc1a0360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ec002a680 tx=0x7f1ec002ceb0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 shutdown_connections 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ecc1035d0 0x7f1ecc1a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ecc102c30 0x7f1ecc1a0360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1eb40b2860 0x7f1eb40b4d70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 --2- 172.21.15.50:0/1222703813 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ecc1022e0 0x7f1ecc19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 >> 172.21.15.50:0/1222703813 conn(0x7f1ecc0fd010 msgr2=0x7f1ecc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 shutdown_connections 2024-05-17T04:49:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:36.418+0000 7f1ed1a78640 1 -- 172.21.15.50:0/1222703813 wait complete. 2024-05-17T04:49:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:36 smithi105 ceph-mon[34795]: pgmap v837: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-05-17T04:49:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:36 smithi110 ceph-mon[35497]: pgmap v837: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-05-17T04:49:36.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:36 smithi050 ceph-mon[31739]: pgmap v837: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.9 MiB/s wr, 323 op/s 2024-05-17T04:49:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 640.0 s, 126.6 tps, lat 802.633 ms stddev 264.610 2024-05-17T04:49:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:38 smithi105 ceph-mon[34795]: pgmap v838: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-17T04:49:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:38 smithi110 ceph-mon[35497]: pgmap v838: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-17T04:49:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:38 smithi050 ceph-mon[31739]: pgmap v838: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-17T04:49:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:40 smithi105 ceph-mon[34795]: pgmap v839: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:49:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:40 smithi110 ceph-mon[35497]: pgmap v839: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:49:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:40 smithi050 ceph-mon[31739]: pgmap v839: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.1 MiB/s wr, 327 op/s 2024-05-17T04:49:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:42 smithi105 ceph-mon[34795]: pgmap v840: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-05-17T04:49:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:42 smithi110 ceph-mon[35497]: pgmap v840: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-05-17T04:49:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:42 smithi050 ceph-mon[31739]: pgmap v840: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-05-17T04:49:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 645.0 s, 120.2 tps, lat 844.720 ms stddev 232.985 2024-05-17T04:49:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:44 smithi105 ceph-mon[34795]: pgmap v841: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-05-17T04:49:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:44 smithi110 ceph-mon[35497]: pgmap v841: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-05-17T04:49:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:44 smithi050 ceph-mon[31739]: pgmap v841: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.7 MiB/s wr, 396 op/s 2024-05-17T04:49:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:46 smithi105 ceph-mon[34795]: pgmap v842: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:49:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:46 smithi110 ceph-mon[35497]: pgmap v842: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:49:46.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:46 smithi050 ceph-mon[31739]: pgmap v842: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:49:47.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:47 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:49:47.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:47 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:49:47.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:47 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:49:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 650.0 s, 104.0 tps, lat 942.190 ms stddev 227.797 2024-05-17T04:49:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:48 smithi105 ceph-mon[34795]: pgmap v843: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:49:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:48 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:49:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:48 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:49:48.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:48 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:49:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:48 smithi110 ceph-mon[35497]: pgmap v843: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:49:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:48 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:49:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:48 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:49:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:48 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:49:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:48 smithi050 ceph-mon[31739]: pgmap v843: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:49:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:48 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:49:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:48 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:49:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:48 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:49:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.581+0000 7f2d607c2640 1 -- 172.21.15.50:0/3575024411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 msgr2=0x7f2d581035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.581+0000 7f2d607c2640 1 --2- 172.21.15.50:0/3575024411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d581035a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009900 tx=0x7f2d5402a410 comp rx=0 tx=0).stop 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 -- 172.21.15.50:0/3575024411 shutdown_connections 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 --2- 172.21.15.50:0/3575024411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d58104c00 0x7f2d5810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 --2- 172.21.15.50:0/3575024411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d581042b0 0x7f2d581046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 --2- 172.21.15.50:0/3575024411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d581035a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 -- 172.21.15.50:0/3575024411 >> 172.21.15.50:0/3575024411 conn(0x7f2d580fe9b0 msgr2=0x7f2d58100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 -- 172.21.15.50:0/3575024411 shutdown_connections 2024-05-17T04:49:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.582+0000 7f2d607c2640 1 -- 172.21.15.50:0/3575024411 wait complete. 2024-05-17T04:49:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 Processor -- start 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 -- start start 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d58197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 0x7f2d58197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 0x7f2d5819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810c9d0 con 0x7f2d581042b0 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810c850 con 0x7f2d58103190 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.583+0000 7f2d607c2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810cb50 con 0x7f2d58104c00 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 0x7f2d5819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5dd5d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 0x7f2d58197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d58197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5dd5d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 0x7f2d58197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52654/0 (socket says 172.21.15.50:52654) 2024-05-17T04:49:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/577883174 learned_addr learned my addr 172.21.15.50:0/577883174 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 -- 172.21.15.50:0/577883174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 msgr2=0x7f2d58197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d58197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 -- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 msgr2=0x7f2d58197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 0x7f2d58197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.584+0000 7f2d5ed5f640 1 -- 172.21.15.50:0/577883174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5806c890 con 0x7f2d58104c00 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.585+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 0x7f2d5819eae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009fd0 tx=0x7f2d5000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.586+0000 7f2d4f7fe640 1 -- 172.21.15.50:0/577883174 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000ce20 con 0x7f2d58104c00 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.586+0000 7f2d4f7fe640 1 -- 172.21.15.50:0/577883174 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d50005020 con 0x7f2d58104c00 2024-05-17T04:49:49.586 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.586+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5806cb80 con 0x7f2d58104c00 2024-05-17T04:49:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.586+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5806d0d0 con 0x7f2d58104c00 2024-05-17T04:49:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.586+0000 7f2d4f7fe640 1 -- 172.21.15.50:0/577883174 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50010690 con 0x7f2d58104c00 2024-05-17T04:49:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.590+0000 7f2d6767e740 1 Processor -- start 2024-05-17T04:49:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.590+0000 7f2d4f7fe640 1 -- 172.21.15.50:0/577883174 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d50010830 con 0x7f2d58104c00 2024-05-17T04:49:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 -- start start 2024-05-17T04:49:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdbf1cc0 con 0x55e4bd8b32f0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdbe1860 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52658/0 (socket says 172.21.15.50:52658) 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d5e55e640 1 -- 172.21.15.50:0/2266912755 learned_addr learned my addr 172.21.15.50:0/2266912755 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdbe58a0 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.591+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/2266912755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 msgr2=0x55e4bdc86bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc86bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/2266912755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 msgr2=0x55e4bdc7fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/2266912755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bdba9d50 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc7f710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d581986b0 tx=0x7f2d48009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb933a390f95fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d4cff9640 1 -- 172.21.15.50:0/2266912755 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48013170 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d4cff9640 1 -- 172.21.15.50:0/2266912755 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d480054b0 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.592+0000 7f2d4cff9640 1 -- 172.21.15.50:0/2266912755 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004ca0 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.593+0000 7f2d4f7fe640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 0x7f2d440b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.593+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 0x7f2d440b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.593+0000 7f2d4f7fe640 1 -- 172.21.15.50:0/577883174 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d500e2c00 con 0x7f2d58104c00 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.593+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 msgr2=0x55e4bdc7f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.593+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc7f710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d581986b0 tx=0x7f2d48009570 comp rx=0 tx=0).stop 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 0x7f2d440b92d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55e4bdc80b10 tx=0x7f2d54031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 shutdown_connections 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc86bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2266912755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc7f710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 >> 172.21.15.50:0/2266912755 conn(0x55e4bd8d0300 msgr2=0x55e4bdc7b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 shutdown_connections 2024-05-17T04:49:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.594+0000 7f2d6767e740 1 -- 172.21.15.50:0/2266912755 wait complete. 2024-05-17T04:49:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.595+0000 7f2d6767e740 1 -- start start 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.595+0000 7f2d6767e740 1 Processor -- start 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdb1dab0 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdb9e4d0 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdc87630 con 0x55e4bd8b32f0 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51958/0 (socket says 172.21.15.50:51958) 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 -- 172.21.15.50:0/2782549512 learned_addr learned my addr 172.21.15.50:0/2782549512 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d6767e740 10 client.0 _ll_register_callbacks cb 0x55e4bdbdc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 -- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 msgr2=0x55e4bdc90590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 -- 172.21.15.50:0/2782549512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8bb4d0 msgr2=0x55e4bdc89630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.596+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc89630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.597+0000 7f2d5e55e640 1 -- 172.21.15.50:0/2782549512 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bdb9f340 con 0x55e4bd8b32f0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.597+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009230 tx=0x7f2d54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1162e696a53975 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.597+0000 7f2d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:49:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54035040 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d54003e40 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d480054b0 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2d54031500 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d6767e740 10 client.26086 fetch_fsmap learned FSMap version 41 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e4bd947790 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.598+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5404c070 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2d54044030 con 0x55e4bd8b32f0 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 10 client.26086 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 20 client.26086 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd89ad80 0x55e4bdc968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 4 client.26086 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2d480054b0 con 0x55e4bd89ad80 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.599+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd89ad80 0x55e4bdc968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.600+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd89ad80 0x55e4bdc968f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5219fba70785894 server_cookie=dd00510c8b94464b in_seq=0 out_seq=0 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.600+0000 7f2d41ffb640 10 client.26086 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.600+0000 7f2d41ffb640 1 -- 172.21.15.50:0/2782549512 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2d480054b0 con 0x55e4bd89ad80 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d41ffb640 10 client.26086 handle_command_reply: tid=0 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 client.26086 shutdown 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 msgr2=0x55e4bdc890f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009230 tx=0x7f2d54009260 comp rx=0 tx=0).stop 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 shutdown_connections 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd89ad80 0x55e4bdc968f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc89630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 --2- 172.21.15.50:0/2782549512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc890f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 >> 172.21.15.50:0/2782549512 conn(0x55e4bd8d0300 msgr2=0x55e4bdc7dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d41ffb640 0 client.26086 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 shutdown_connections 2024-05-17T04:49:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 1 -- 172.21.15.50:0/2782549512 wait complete. 2024-05-17T04:49:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.601+0000 7f2d6767e740 20 client.26086 trim_cache size 0 max 16384 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.622+0000 7f2d6767e740 1 Processor -- start 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.622+0000 7f2d6767e740 1 -- start start 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.622+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.622+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.622+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d6767e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581b4e70 con 0x55e4bd8b32f0 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d6767e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581b2740 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d6767e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58074bc0 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:51960/0 (socket says 172.21.15.50:51960) 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/870343204 learned_addr learned my addr 172.21.15.50:0/870343204 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/870343204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 msgr2=0x55e4bdc90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/870343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 msgr2=0x55e4bdc7b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 -- 172.21.15.50:0/870343204 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bd9c0620 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.623+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d581986b0 tx=0x7f2d4801c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8846d7dca4f29291 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.624+0000 7f2d43fff640 1 -- 172.21.15.50:0/870343204 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48009880 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.624+0000 7f2d43fff640 1 -- 172.21.15.50:0/870343204 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d48032070 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.624+0000 7f2d43fff640 1 -- 172.21.15.50:0/870343204 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800fbf0 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- 172.21.15.50:0/870343204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 msgr2=0x55e4bdc90520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d581986b0 tx=0x7f2d4801c040 comp rx=0 tx=0).stop 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- 172.21.15.50:0/870343204 shutdown_connections 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc90520 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 --2- 172.21.15.50:0/870343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc7b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- 172.21.15.50:0/870343204 >> 172.21.15.50:0/870343204 conn(0x55e4bd8d0300 msgr2=0x55e4bdc7aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- 172.21.15.50:0/870343204 shutdown_connections 2024-05-17T04:49:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- 172.21.15.50:0/870343204 wait complete. 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 -- start start 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.625+0000 7f2d6767e740 1 Processor -- start 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.626+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.626+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d6767e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52678/0 (socket says 172.21.15.50:52678) 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 -- 172.21.15.50:0/597493595 learned_addr learned my addr 172.21.15.50:0/597493595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5dd5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d6767e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581b0c00 con 0x55e4bd8b32f0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581b2070 con 0x55e4bd8dc1b0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d581b12d0 con 0x55e4bd8bb4d0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d6767e740 10 client.0 _ll_register_callbacks cb 0x7f2d54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 -- 172.21.15.50:0/597493595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 msgr2=0x55e4bdc8d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc8d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 -- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 msgr2=0x55e4bdc8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bdc90cc0 con 0x55e4bd8b32f0 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5ed5f640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d5e55e640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d5403d9b0 tx=0x7f2d54005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467734765e4edafc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.627+0000 7f2d41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:49:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54035070 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2d54031ab0 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d48032070 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2d54004330 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 10 client.16365 fetch_fsmap learned FSMap version 41 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e4bdcfdb40 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d48032070 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2d5405c070 con 0x55e4bd8b32f0 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 10 client.16365 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.628+0000 7f2d6767e740 20 client.16365 populate_metadata read hostname 'smithi050' 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd8701c0 0x55e4bdd005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d6767e740 4 client.16365 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:49:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4bdbf3c10 con 0x55e4bd8701c0 2024-05-17T04:49:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd8701c0 0x55e4bdd005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:49:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d41ffb640 10 client.16365 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.629+0000 7f2d5dd5d640 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd8701c0 0x55e4bdd005c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8443271298a7a1b server_cookie=763c57838e3b5809 in_seq=0 out_seq=0 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d41ffb640 1 -- 172.21.15.50:0/597493595 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4bdbf3c10 con 0x55e4bd8701c0 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d41ffb640 10 client.16365 handle_command_reply: tid=0 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 client.16365 shutdown 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 msgr2=0x55e4bdc90620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d5403d9b0 tx=0x7f2d54005720 comp rx=0 tx=0).stop 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 shutdown_connections 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e4bd8dc1b0 0x55e4bdc8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4bd8bb4d0 0x55e4bdc8d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55e4bd8701c0 0x55e4bdd005c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 --2- 172.21.15.50:0/597493595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4bd8b32f0 0x55e4bdc90620 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d41ffb640 0 client.16365 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.655+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 >> 172.21.15.50:0/597493595 conn(0x55e4bd8d0300 msgr2=0x55e4bdc797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 shutdown_connections 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d6767e740 1 -- 172.21.15.50:0/597493595 wait complete. 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d6767e740 20 client.16365 trim_cache size 0 max 16384 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:49:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 msgr2=0x7f2d440b92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 0x7f2d440b92d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55e4bdc80b10 tx=0x7f2d54031400 comp rx=0 tx=0).stop 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 msgr2=0x7f2d5819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.656+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 0x7f2d5819eae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009fd0 tx=0x7f2d5000ef10 comp rx=0 tx=0).stop 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 shutdown_connections 2024-05-17T04:49:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58104c00 0x7f2d5819eae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f2d440b6dc0 0x7f2d440b92d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d581042b0 0x7f2d58197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 --2- 172.21.15.50:0/577883174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d58103190 0x7f2d58197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 >> 172.21.15.50:0/577883174 conn(0x7f2d580fe9b0 msgr2=0x7f2d58107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 shutdown_connections 2024-05-17T04:49:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:49:49.657+0000 7f2d607c2640 1 -- 172.21.15.50:0/577883174 wait complete. 2024-05-17T04:49:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:50 smithi105 ceph-mon[34795]: pgmap v844: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.5 MiB/s wr, 349 op/s 2024-05-17T04:49:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:50 smithi110 ceph-mon[35497]: pgmap v844: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.5 MiB/s wr, 349 op/s 2024-05-17T04:49:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:50 smithi050 ceph-mon[31739]: pgmap v844: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.5 MiB/s wr, 349 op/s 2024-05-17T04:49:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:52 smithi105 ceph-mon[34795]: pgmap v845: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-05-17T04:49:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:52 smithi110 ceph-mon[35497]: pgmap v845: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-05-17T04:49:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:52 smithi050 ceph-mon[31739]: pgmap v845: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-05-17T04:49:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 655.0 s, 107.4 tps, lat 951.058 ms stddev 237.057 2024-05-17T04:49:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:54 smithi105 ceph-mon[34795]: pgmap v846: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:54 smithi110 ceph-mon[35497]: pgmap v846: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:54 smithi050 ceph-mon[31739]: pgmap v846: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.5 MiB/s wr, 385 op/s 2024-05-17T04:49:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:56 smithi105 ceph-mon[34795]: pgmap v847: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:49:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:56 smithi110 ceph-mon[35497]: pgmap v847: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:49:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:56 smithi050 ceph-mon[31739]: pgmap v847: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:49:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 660.0 s, 127.0 tps, lat 797.022 ms stddev 281.516 2024-05-17T04:49:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:49:58 smithi105 ceph-mon[34795]: pgmap v848: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.3 MiB/s wr, 363 op/s 2024-05-17T04:49:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:49:58 smithi110 ceph-mon[35497]: pgmap v848: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.3 MiB/s wr, 363 op/s 2024-05-17T04:49:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:49:58 smithi050 ceph-mon[31739]: pgmap v848: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 2.3 MiB/s wr, 363 op/s 2024-05-17T04:50:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:00 smithi105 ceph-mon[34795]: pgmap v849: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.2 MiB/s wr, 340 op/s 2024-05-17T04:50:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:00 smithi105 ceph-mon[34795]: overall HEALTH_OK 2024-05-17T04:50:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:00 smithi110 ceph-mon[35497]: pgmap v849: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.2 MiB/s wr, 340 op/s 2024-05-17T04:50:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:00 smithi110 ceph-mon[35497]: overall HEALTH_OK 2024-05-17T04:50:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:00 smithi050 ceph-mon[31739]: pgmap v849: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.2 MiB/s wr, 340 op/s 2024-05-17T04:50:00.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:00 smithi050 ceph-mon[31739]: overall HEALTH_OK 2024-05-17T04:50:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:02 smithi105 ceph-mon[34795]: pgmap v850: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-05-17T04:50:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:02 smithi110 ceph-mon[35497]: pgmap v850: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-05-17T04:50:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:02 smithi050 ceph-mon[31739]: pgmap v850: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 344 op/s 2024-05-17T04:50:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 665.0 s, 107.8 tps, lat 905.683 ms stddev 219.394 2024-05-17T04:50:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f159ffff640 1 -- 172.21.15.50:0/614006466 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a002f340 con 0x7f15a8102080 2024-05-17T04:50:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 -- 172.21.15.50:0/614006466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8102080 msgr2=0x7f15a8102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 --2- 172.21.15.50:0/614006466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8102080 0x7f15a8102490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009ca0 tx=0x7f15a002a8f0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 -- 172.21.15.50:0/614006466 shutdown_connections 2024-05-17T04:50:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 --2- 172.21.15.50:0/614006466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a8103af0 0x7f15a810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 --2- 172.21.15.50:0/614006466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a81031a0 0x7f15a81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 --2- 172.21.15.50:0/614006466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8102080 0x7f15a8102490 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.808+0000 7f15af2d1640 1 -- 172.21.15.50:0/614006466 >> 172.21.15.50:0/614006466 conn(0x7f15a80fd880 msgr2=0x7f15a80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.809+0000 7f15af2d1640 1 -- 172.21.15.50:0/614006466 shutdown_connections 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.809+0000 7f15af2d1640 1 -- 172.21.15.50:0/614006466 wait complete. 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 Processor -- start 2024-05-17T04:50:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 -- start start 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a8102080 0x7f15a8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a81031a0 0x7f15a8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810ecc0 con 0x7f15a8102080 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810eb40 con 0x7f15a8103af0 2024-05-17T04:50:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.810+0000 7f15af2d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810ee40 con 0x7f15a81031a0 2024-05-17T04:50:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:35872/0 (socket says 172.21.15.50:35872) 2024-05-17T04:50:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 -- 172.21.15.50:0/2181221684 learned_addr learned my addr 172.21.15.50:0/2181221684 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad06d640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a8102080 0x7f15a8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ac86c640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a81031a0 0x7f15a8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 -- 172.21.15.50:0/2181221684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a81031a0 msgr2=0x7f15a8199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a81031a0 0x7f15a8199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 -- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a8102080 msgr2=0x7f15a8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a8102080 0x7f15a8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.811+0000 7f15ad86e640 1 -- 172.21.15.50:0/2181221684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a81a1560 con 0x7f15a8103af0 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.812+0000 7f15ad86e640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f159800b3f0 tx=0x7f159800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.812+0000 7f159e7fc640 1 -- 172.21.15.50:0/2181221684 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598015020 con 0x7f15a8103af0 2024-05-17T04:50:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.812+0000 7f159e7fc640 1 -- 172.21.15.50:0/2181221684 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1598011cc0 con 0x7f15a8103af0 2024-05-17T04:50:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.813+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a81a1850 con 0x7f15a8103af0 2024-05-17T04:50:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.813+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81a1e20 con 0x7f15a8103af0 2024-05-17T04:50:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.813+0000 7f159e7fc640 1 -- 172.21.15.50:0/2181221684 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598010ce0 con 0x7f15a8103af0 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.816+0000 7f15b6183740 1 Processor -- start 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.816+0000 7f159e7fc640 1 -- 172.21.15.50:0/2181221684 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1598018020 con 0x7f15a8103af0 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.816+0000 7f15b6183740 1 -- start start 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5555c9741cc0 con 0x5555c94032f0 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c9731860 con 0x5555c942c1b0 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15b6183740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5555c97358a0 con 0x5555c940b4d0 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39544/0 (socket says 172.21.15.50:39544) 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 -- 172.21.15.50:0/578701418 learned_addr learned my addr 172.21.15.50:0/578701418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ac86c640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ac86c640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 -- 172.21.15.50:0/578701418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 msgr2=0x5555c97df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 -- 172.21.15.50:0/578701418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 msgr2=0x5555c97d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ad06d640 1 -- 172.21.15.50:0/578701418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c96f9d50 con 0x5555c94032f0 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.817+0000 7f15ac86c640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.818+0000 7f159e7fc640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 0x7f15940b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.818+0000 7f15ad06d640 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15a81030a0 tx=0x7f15a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eed055bfa7b941 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.818+0000 7f15ac86c640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 0x7f15940b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.818+0000 7f159e7fc640 1 -- 172.21.15.50:0/2181221684 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15980e1d10 con 0x7f15a8103af0 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.818+0000 7f15937fe640 1 -- 172.21.15.50:0/578701418 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0031e00 con 0x5555c94032f0 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.819+0000 7f15937fe640 1 -- 172.21.15.50:0/578701418 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a0036030 con 0x5555c94032f0 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.819+0000 7f15937fe640 1 -- 172.21.15.50:0/578701418 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a00044d0 con 0x5555c94032f0 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.819+0000 7f15b6183740 1 -- 172.21.15.50:0/578701418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 msgr2=0x5555c97d8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.819+0000 7f15ac86c640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 0x7f15940b4d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5555c97d9170 tx=0x7f15a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.819+0000 7f15b6183740 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15a81030a0 tx=0x7f15a0005fd0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 -- 172.21.15.50:0/578701418 shutdown_connections 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97d8500 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 --2- 172.21.15.50:0/578701418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 -- 172.21.15.50:0/578701418 >> 172.21.15.50:0/578701418 conn(0x5555c9420300 msgr2=0x5555c97cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 -- 172.21.15.50:0/578701418 shutdown_connections 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 -- 172.21.15.50:0/578701418 wait complete. 2024-05-17T04:50:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.820+0000 7f15b6183740 1 -- start start 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 Processor -- start 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5555c966dab0 con 0x5555c94032f0 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c96ee4d0 con 0x5555c942c1b0 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5555c97dfe40 con 0x5555c940b4d0 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15b6183740 10 client.0 _ll_register_callbacks cb 0x5555c972c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97e1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:39554/0 (socket says 172.21.15.50:39554) 2024-05-17T04:50:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ad06d640 1 -- 172.21.15.50:0/4061754783 learned_addr learned my addr 172.21.15.50:0/4061754783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ac86c640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ad86e640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ac86c640 1 -- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 msgr2=0x5555c97e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ac86c640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ac86c640 1 -- 172.21.15.50:0/4061754783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 msgr2=0x5555c97e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.821+0000 7f15ac86c640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f15ac86c640 1 -- 172.21.15.50:0/4061754783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c96ef340 con 0x5555c940b4d0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f15ac86c640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e1d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15a401a040 tx=0x7f15a4004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4cbd350700ffd00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f1590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a401e070 con 0x5555c940b4d0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15a4018d30 con 0x5555c940b4d0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.822+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a0036030 con 0x5555c940b4d0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f15a4033050 con 0x5555c940b4d0 2024-05-17T04:50:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 10 client.26101 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5555c9497790 con 0x5555c940b4d0 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a0036030 con 0x5555c940b4d0 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f15a40163d0 con 0x5555c940b4d0 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 10 client.26101 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 10 client.26101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 20 client.26101 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93ead80 0x5555c97ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 4 client.26101 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c9743c10 con 0x5555c93ead80 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.823+0000 7f15ad06d640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93ead80 0x5555c97ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.824+0000 7f1590ff9640 10 client.26101 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.824+0000 7f15ad06d640 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93ead80 0x5555c97ef1d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18eaee4a6bd95c3f server_cookie=6e152426a39ddfca in_seq=0 out_seq=0 2024-05-17T04:50:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f1590ff9640 1 -- 172.21.15.50:0/4061754783 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555c9743c10 con 0x5555c93ead80 2024-05-17T04:50:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f1590ff9640 10 client.26101 handle_command_reply: tid=0 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 client.26101 shutdown 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 msgr2=0x5555c97e1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e1d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15a401a040 tx=0x7f15a4004700 comp rx=0 tx=0).stop 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 shutdown_connections 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c942c1b0 0x5555c97e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e1d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93ead80 0x5555c97ef1d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 --2- 172.21.15.50:0/4061754783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c94032f0 0x5555c97e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 >> 172.21.15.50:0/4061754783 conn(0x5555c9420300 msgr2=0x5555c97ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.825+0000 7f1590ff9640 0 client.26101 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.826+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 shutdown_connections 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.826+0000 7f15b6183740 1 -- 172.21.15.50:0/4061754783 wait complete. 2024-05-17T04:50:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.826+0000 7f15b6183740 20 client.26101 trim_cache size 0 max 16384 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.851+0000 7f15b6183740 1 Processor -- start 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.851+0000 7f15b6183740 1 -- start start 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 0x5555c97ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c97e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:50220/0 (socket says 172.21.15.50:50220) 2024-05-17T04:50:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ac86c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 0x5555c97ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ac86c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 0x5555c97ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:35898/0 (socket says 172.21.15.50:35898) 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ad06d640 1 -- 172.21.15.50:0/1984565370 learned_addr learned my addr 172.21.15.50:0/1984565370 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15ad86e640 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c97e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81ac710 con 0x5555c942c1b0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81a9fe0 con 0x5555c940b4d0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.852+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81a7e30 con 0x5555c94032f0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 -- 172.21.15.50:0/1984565370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 msgr2=0x5555c97ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 0x5555c97ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 -- 172.21.15.50:0/1984565370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 msgr2=0x5555c97e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c97e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 -- 172.21.15.50:0/1984565370 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c9510620 con 0x5555c94032f0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f15ad06d640 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009a60 tx=0x7f15a002a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5acc24bab6079a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f1592ffd640 1 -- 172.21.15.50:0/1984565370 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a004a040 con 0x5555c94032f0 2024-05-17T04:50:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f1592ffd640 1 -- 172.21.15.50:0/1984565370 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a005d070 con 0x5555c94032f0 2024-05-17T04:50:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.853+0000 7f1592ffd640 1 -- 172.21.15.50:0/1984565370 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004540 con 0x5555c94032f0 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.854+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 msgr2=0x5555c97c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.854+0000 7f15b6183740 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009a60 tx=0x7f15a002a8f0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 shutdown_connections 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c97e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c940b4d0 0x5555c97ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 --2- 172.21.15.50:0/1984565370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c94032f0 0x5555c97c9ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 >> 172.21.15.50:0/1984565370 conn(0x5555c9420300 msgr2=0x5555c97c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 shutdown_connections 2024-05-17T04:50:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 -- 172.21.15.50:0/1984565370 wait complete. 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 -- start start 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.855+0000 7f15b6183740 1 Processor -- start 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c9855530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81a84a0 con 0x5555c942c1b0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81a9910 con 0x5555c94032f0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15a81a8b70 con 0x5555c940b4d0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15b6183740 10 client.0 _ll_register_callbacks cb 0x7f15a4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.856+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:35906/0 (socket says 172.21.15.50:35906) 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 -- 172.21.15.50:0/606284890 learned_addr learned my addr 172.21.15.50:0/606284890 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ac86c640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad86e640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c9855530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 -- 172.21.15.50:0/606284890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 msgr2=0x5555c97e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 -- 172.21.15.50:0/606284890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 msgr2=0x5555c9855530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c9855530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 -- 172.21.15.50:0/606284890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c97e9610 con 0x5555c94032f0 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f15ad06d640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009980 tx=0x7f15a0004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52005141dde70f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.857+0000 7f1590ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0066060 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a005d070 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15a00042c0 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f15a006d050 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f15b6183740 10 client.26132 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.858+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5555c9856920 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15a0056070 con 0x5555c94032f0 2024-05-17T04:50:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f15a0043070 con 0x5555c94032f0 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 10 client.26132 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 20 client.26132 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93c01c0 0x5555c9858ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 4 client.26132 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f15a005d070 con 0x5555c93c01c0 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.859+0000 7f15ac86c640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93c01c0 0x5555c9858ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.860+0000 7f15ac86c640 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93c01c0 0x5555c9858ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394c5c65a35ba58f server_cookie=10b404c0727833ee in_seq=0 out_seq=0 2024-05-17T04:50:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.860+0000 7f1590ff9640 10 client.26132 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.882+0000 7f1590ff9640 1 -- 172.21.15.50:0/606284890 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f15a005d070 con 0x5555c93c01c0 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.882+0000 7f1590ff9640 10 client.26132 handle_command_reply: tid=0 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.882+0000 7f15b6183740 1 client.26132 shutdown 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.882+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 msgr2=0x5555c97e60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.882+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009980 tx=0x7f15a0004a80 comp rx=0 tx=0).stop 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 shutdown_connections 2024-05-17T04:50:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5555c93c01c0 0x5555c9858ef0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555c942c1b0 0x5555c9855530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555c940b4d0 0x5555c97e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 --2- 172.21.15.50:0/606284890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c94032f0 0x5555c97e60e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 >> 172.21.15.50:0/606284890 conn(0x5555c9420300 msgr2=0x5555c97ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f1590ff9640 0 client.26132 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 shutdown_connections 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 1 -- 172.21.15.50:0/606284890 wait complete. 2024-05-17T04:50:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.883+0000 7f15b6183740 20 client.26132 trim_cache size 0 max 16384 2024-05-17T04:50:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:50:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:50:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 msgr2=0x7f15940b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 0x7f15940b4d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5555c97d9170 tx=0x7f15a4008040 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 msgr2=0x7f15a81a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f159800b3f0 tx=0x7f159800b8b0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 shutdown_connections 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a8103af0 0x7f15a81a0ef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15a81031a0 0x7f15a8199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f15940b2870 0x7f15940b4d80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 --2- 172.21.15.50:0/2181221684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15a8102080 0x7f15a8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 >> 172.21.15.50:0/2181221684 conn(0x7f15a80fd880 msgr2=0x7f15a8106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 shutdown_connections 2024-05-17T04:50:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:02.884+0000 7f15af2d1640 1 -- 172.21.15.50:0/2181221684 wait complete. 2024-05-17T04:50:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:04 smithi105 ceph-mon[34795]: pgmap v851: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.3 MiB/s wr, 383 op/s 2024-05-17T04:50:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:04 smithi110 ceph-mon[35497]: pgmap v851: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.3 MiB/s wr, 383 op/s 2024-05-17T04:50:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:04 smithi050 ceph-mon[31739]: pgmap v851: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.3 MiB/s wr, 383 op/s 2024-05-17T04:50:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:06 smithi105 ceph-mon[34795]: pgmap v852: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-05-17T04:50:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:06 smithi110 ceph-mon[35497]: pgmap v852: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-05-17T04:50:06.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:06 smithi050 ceph-mon[31739]: pgmap v852: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-05-17T04:50:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 670.0 s, 94.4 tps, lat 1043.203 ms stddev 253.487 2024-05-17T04:50:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:08 smithi105 ceph-mon[34795]: pgmap v853: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-05-17T04:50:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:08 smithi110 ceph-mon[35497]: pgmap v853: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-05-17T04:50:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:08 smithi050 ceph-mon[31739]: pgmap v853: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-05-17T04:50:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:10 smithi105 ceph-mon[34795]: pgmap v854: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:50:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:10 smithi110 ceph-mon[35497]: pgmap v854: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:50:10.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:10 smithi050 ceph-mon[31739]: pgmap v854: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:50:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:12 smithi105 ceph-mon[34795]: pgmap v855: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-17T04:50:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:12 smithi110 ceph-mon[35497]: pgmap v855: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-17T04:50:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:12 smithi050 ceph-mon[31739]: pgmap v855: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-05-17T04:50:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 675.0 s, 114.6 tps, lat 895.563 ms stddev 214.239 2024-05-17T04:50:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:14 smithi105 ceph-mon[34795]: pgmap v856: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-05-17T04:50:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:14 smithi110 ceph-mon[35497]: pgmap v856: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-05-17T04:50:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:14 smithi050 ceph-mon[31739]: pgmap v856: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.9 MiB/s wr, 340 op/s 2024-05-17T04:50:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 -- 172.21.15.50:0/2251677421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e01031a0 msgr2=0x7f79e01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 --2- 172.21.15.50:0/2251677421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e01031a0 0x7f79e01035b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009900 tx=0x7f79d402a410 comp rx=0 tx=0).stop 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 -- 172.21.15.50:0/2251677421 shutdown_connections 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 --2- 172.21.15.50:0/2251677421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0103af0 0x7f79e010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 --2- 172.21.15.50:0/2251677421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e01031a0 0x7f79e01035b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 --2- 172.21.15.50:0/2251677421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e0102080 0x7f79e0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.018+0000 7f79e604d640 1 -- 172.21.15.50:0/2251677421 >> 172.21.15.50:0/2251677421 conn(0x7f79e00fd880 msgr2=0x7f79e00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.019+0000 7f79e604d640 1 -- 172.21.15.50:0/2251677421 shutdown_connections 2024-05-17T04:50:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.019+0000 7f79e604d640 1 -- 172.21.15.50:0/2251677421 wait complete. 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.019+0000 7f79e604d640 1 Processor -- start 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.019+0000 7f79e604d640 1 -- start start 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e0103af0 0x7f79e019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e01a1090 0x7f79e01a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79e010dab0 con 0x7f79e0103af0 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79e010d930 con 0x7f79e0102080 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79e604d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79e010dc30 con 0x7f79e01a1090 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53600/0 (socket says 172.21.15.50:53600) 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 -- 172.21.15.50:0/3999318787 learned_addr learned my addr 172.21.15.50:0/3999318787 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79dffff640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e01a1090 0x7f79e01a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79deffd640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e0103af0 0x7f79e019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 -- 172.21.15.50:0/3999318787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e01a1090 msgr2=0x7f79e01a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e01a1090 0x7f79e01a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 -- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e0103af0 msgr2=0x7f79e019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e0103af0 0x7f79e019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.020+0000 7f79df7fe640 1 -- 172.21.15.50:0/3999318787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e01a3be0 con 0x7f79e0102080 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79df7fe640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79d000e9f0 tx=0x7f79d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79dcff9640 1 -- 172.21.15.50:0/3999318787 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d000ccc0 con 0x7f79e0102080 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e01a3ed0 con 0x7f79e0102080 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79dcff9640 1 -- 172.21.15.50:0/3999318787 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79d00054b0 con 0x7f79e0102080 2024-05-17T04:50:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01a4330 con 0x7f79e0102080 2024-05-17T04:50:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.021+0000 7f79dcff9640 1 -- 172.21.15.50:0/3999318787 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0010430 con 0x7f79e0102080 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.024+0000 7f79dcff9640 1 -- 172.21.15.50:0/3999318787 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f79d00105d0 con 0x7f79e0102080 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 Processor -- start 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 -- start start 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12913cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e12885cc0 con 0x562e125701b0 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e12875860 con 0x562e1254f4d0 2024-05-17T04:50:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e128798a0 con 0x562e125472f0 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12913cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40386/0 (socket says 172.21.15.50:40386) 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79dffff640 1 -- 172.21.15.50:0/2724208182 learned_addr learned my addr 172.21.15.50:0/2724208182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.025+0000 7f79deffd640 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 -- 172.21.15.50:0/2724208182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 msgr2=0x562e12913cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12913cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 -- 172.21.15.50:0/2724208182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 msgr2=0x562e1291ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 -- 172.21.15.50:0/2724208182 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1283dd50 con 0x562e125701b0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dffff640 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009f50 tx=0x7f79d80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e85ae34dacb33ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79dcff9640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 0x7f79c80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79deffd640 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79c5ffb640 1 -- 172.21.15.50:0/2724208182 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8004d20 con 0x562e125701b0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79c5ffb640 1 -- 172.21.15.50:0/2724208182 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79d8004050 con 0x562e125701b0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79c5ffb640 1 -- 172.21.15.50:0/2724208182 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8005260 con 0x562e125701b0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.026+0000 7f79df7fe640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 0x7f79c80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.027+0000 7f79dcff9640 1 -- 172.21.15.50:0/3999318787 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f79d00e28f0 con 0x7f79e0102080 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79df7fe640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 0x7f79c80b4d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79d00a9ab0 tx=0x7f79d00e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 -- 172.21.15.50:0/2724208182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 msgr2=0x562e12913770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009f50 tx=0x7f79d80118d0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 -- 172.21.15.50:0/2724208182 shutdown_connections 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12913cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/2724208182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12913770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 -- 172.21.15.50:0/2724208182 >> 172.21.15.50:0/2724208182 conn(0x562e12564300 msgr2=0x562e1290f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 -- 172.21.15.50:0/2724208182 shutdown_connections 2024-05-17T04:50:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.028+0000 7f79ecfe0740 1 -- 172.21.15.50:0/2724208182 wait complete. 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.029+0000 7f79ecfe0740 1 -- start start 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.029+0000 7f79ecfe0740 1 Processor -- start 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125701b0 0x562e12924560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e127b1ab0 con 0x562e125472f0 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e128324d0 con 0x562e1254f4d0 2024-05-17T04:50:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562e1291b690 con 0x562e125701b0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40388/0 (socket says 172.21.15.50:40388) 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.030+0000 7f79dffff640 1 -- 172.21.15.50:0/1139019390 learned_addr learned my addr 172.21.15.50:0/1139019390 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79ecfe0740 10 client.0 _ll_register_callbacks cb 0x562e12870950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79deffd640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125701b0 0x562e12924560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 -- 172.21.15.50:0/1139019390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125701b0 msgr2=0x562e12924560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125701b0 0x562e12924560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 -- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 msgr2=0x562e1291d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.031+0000 7f79dffff640 1 -- 172.21.15.50:0/1139019390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e12833340 con 0x562e125472f0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79dffff640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009f20 tx=0x7f79d8009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36fe363ffb69bbe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79beffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d802a020 con 0x562e125472f0 2024-05-17T04:50:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f79d8005240 con 0x562e125472f0 2024-05-17T04:50:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d8004050 con 0x562e125472f0 2024-05-17T04:50:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f79d8003e70 con 0x562e125472f0 2024-05-17T04:50:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79ecfe0740 10 client.16392 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.032+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e125db790 con 0x562e125472f0 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79d8004050 con 0x562e125472f0 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f79d8025910 con 0x562e125472f0 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 10 client.16392 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 20 client.16392 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e1252ed80 0x562e1292a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 4 client.16392 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e12887c10 con 0x562e1252ed80 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.033+0000 7f79deffd640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e1252ed80 0x562e1292a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.034+0000 7f79deffd640 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e1252ed80 0x562e1292a670 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b27a08b409d3f56 server_cookie=6e5faf135e414f99 in_seq=0 out_seq=0 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.034+0000 7f79beffd640 10 client.16392 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79beffd640 1 -- 172.21.15.50:0/1139019390 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e12887c10 con 0x562e1252ed80 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79beffd640 10 client.16392 handle_command_reply: tid=0 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 client.16392 shutdown 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 msgr2=0x562e1291d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009f20 tx=0x7f79d8009be0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 shutdown_connections 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125701b0 0x562e12924560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e1291d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e1252ed80 0x562e1292a670 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1139019390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125472f0 0x562e1291d0c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 >> 172.21.15.50:0/1139019390 conn(0x562e12564300 msgr2=0x562e12912010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79beffd640 0 client.16392 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 shutdown_connections 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1139019390 wait complete. 2024-05-17T04:50:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.035+0000 7f79ecfe0740 20 client.16392 trim_cache size 0 max 16384 2024-05-17T04:50:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.057+0000 7f79ecfe0740 1 Processor -- start 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.057+0000 7f79ecfe0740 1 -- start start 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e1254f4d0 0x562e12911b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12912090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01b1240 con 0x562e125701b0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01aeb10 con 0x562e125472f0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01ac960 con 0x562e1254f4d0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:53626/0 (socket says 172.21.15.50:53626) 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 -- 172.21.15.50:0/884193935 learned_addr learned my addr 172.21.15.50:0/884193935 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79deffd640 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e1254f4d0 0x562e12911b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12912090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 -- 172.21.15.50:0/884193935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e1254f4d0 msgr2=0x562e12911b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e1254f4d0 0x562e12911b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 -- 172.21.15.50:0/884193935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 msgr2=0x562e12912090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12912090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.058+0000 7f79dffff640 1 -- 172.21.15.50:0/884193935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e12654620 con 0x562e125472f0 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.059+0000 7f79dffff640 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79d8005d70 tx=0x7f79d8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1274dee52494f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.059+0000 7f79c57fa640 1 -- 172.21.15.50:0/884193935 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d802a040 con 0x562e125472f0 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.059+0000 7f79c57fa640 1 -- 172.21.15.50:0/884193935 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79d8004e20 con 0x562e125472f0 2024-05-17T04:50:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.059+0000 7f79c57fa640 1 -- 172.21.15.50:0/884193935 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80256a0 con 0x562e125472f0 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 -- 172.21.15.50:0/884193935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 msgr2=0x562e1290f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79d8005d70 tx=0x7f79d8000f40 comp rx=0 tx=0).stop 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 -- 172.21.15.50:0/884193935 shutdown_connections 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e1254f4d0 0x562e12911b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/884193935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e125472f0 0x562e1290f2e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.060+0000 7f79ecfe0740 1 -- 172.21.15.50:0/884193935 >> 172.21.15.50:0/884193935 conn(0x562e12564300 msgr2=0x562e1290ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.061+0000 7f79ecfe0740 1 -- 172.21.15.50:0/884193935 shutdown_connections 2024-05-17T04:50:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.061+0000 7f79ecfe0740 1 -- 172.21.15.50:0/884193935 wait complete. 2024-05-17T04:50:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.061+0000 7f79ecfe0740 1 -- start start 2024-05-17T04:50:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.061+0000 7f79ecfe0740 1 Processor -- start 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12923b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e129206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01acfd0 con 0x562e125701b0 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01ae440 con 0x562e1254f4d0 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01ad6a0 con 0x562e125472f0 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79ecfe0740 10 client.0 _ll_register_callbacks cb 0x7f79d00e8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40410/0 (socket says 172.21.15.50:40410) 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 -- 172.21.15.50:0/1347242389 learned_addr learned my addr 172.21.15.50:0/1347242389 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12923b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79deffd640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e129206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 -- 172.21.15.50:0/1347242389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 msgr2=0x562e12923b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12923b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 -- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 msgr2=0x562e129206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e129206b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.062+0000 7f79dffff640 1 -- 172.21.15.50:0/1347242389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e129909f0 con 0x562e125701b0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79dffff640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562e1291fc40 tx=0x7f79d8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841f103e964d00a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79beffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d802a040 con 0x562e125701b0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d8004e20 con 0x562e125701b0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f79d800fa20 con 0x562e125701b0 2024-05-17T04:50:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.063+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f79d804b050 con 0x562e125701b0 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 10 client.16398 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562e12991c50 con 0x562e125701b0 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79d8004e20 con 0x562e125701b0 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f79d8005160 con 0x562e125701b0 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 10 client.16398 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 20 client.16398 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e125041c0 0x562e12994450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 4 client.16398 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e128813f0 con 0x562e125041c0 2024-05-17T04:50:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.064+0000 7f79deffd640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e125041c0 0x562e12994450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.065+0000 7f79deffd640 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e125041c0 0x562e12994450 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebba632684a4346 server_cookie=94de56ca4eb11a48 in_seq=0 out_seq=0 2024-05-17T04:50:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.065+0000 7f79beffd640 10 client.16398 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.097+0000 7f79beffd640 1 -- 172.21.15.50:0/1347242389 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562e128813f0 con 0x562e125041c0 2024-05-17T04:50:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.097+0000 7f79beffd640 10 client.16398 handle_command_reply: tid=0 2024-05-17T04:50:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.097+0000 7f79ecfe0740 1 client.16398 shutdown 2024-05-17T04:50:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 msgr2=0x562e12920c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562e1291fc40 tx=0x7f79d8009fd0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 shutdown_connections 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562e125041c0 0x562e12994450 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e125701b0 0x562e12920c70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79beffd640 0 client.16398 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e1254f4d0 0x562e129206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 --2- 172.21.15.50:0/1347242389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e125472f0 0x562e12923b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 >> 172.21.15.50:0/1347242389 conn(0x562e12564300 msgr2=0x562e12912510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 shutdown_connections 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 1 -- 172.21.15.50:0/1347242389 wait complete. 2024-05-17T04:50:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.098+0000 7f79ecfe0740 20 client.16398 trim_cache size 0 max 16384 2024-05-17T04:50:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:50:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:50:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:50:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 msgr2=0x7f79c80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 0x7f79c80b4d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f79d00a9ab0 tx=0x7f79d00e8530 comp rx=0 tx=0).stop 2024-05-17T04:50:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 msgr2=0x7f79e019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79d000e9f0 tx=0x7f79d000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 shutdown_connections 2024-05-17T04:50:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79e01a1090 0x7f79e01a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f79c80b2860 0x7f79c80b4d70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79e0103af0 0x7f79e019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 --2- 172.21.15.50:0/3999318787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79e0102080 0x7f79e019bbd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 >> 172.21.15.50:0/3999318787 conn(0x7f79e00fd880 msgr2=0x7f79e00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 shutdown_connections 2024-05-17T04:50:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:16.099+0000 7f79e604d640 1 -- 172.21.15.50:0/3999318787 wait complete. 2024-05-17T04:50:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:16 smithi105 ceph-mon[34795]: pgmap v857: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-05-17T04:50:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:16 smithi110 ceph-mon[35497]: pgmap v857: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-05-17T04:50:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:16 smithi050 ceph-mon[31739]: pgmap v857: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 2.3 MiB/s wr, 337 op/s 2024-05-17T04:50:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 680.0 s, 97.8 tps, lat 978.581 ms stddev 221.892 2024-05-17T04:50:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:18 smithi105 ceph-mon[34795]: pgmap v858: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-05-17T04:50:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:18 smithi110 ceph-mon[35497]: pgmap v858: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-05-17T04:50:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:18 smithi050 ceph-mon[31739]: pgmap v858: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-05-17T04:50:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:20 smithi105 ceph-mon[34795]: pgmap v859: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:50:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:20 smithi110 ceph-mon[35497]: pgmap v859: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:50:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:20 smithi050 ceph-mon[31739]: pgmap v859: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.2 MiB/s wr, 335 op/s 2024-05-17T04:50:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:22 smithi105 ceph-mon[34795]: pgmap v860: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.9 MiB/s wr, 324 op/s 2024-05-17T04:50:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:22 smithi110 ceph-mon[35497]: pgmap v860: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.9 MiB/s wr, 324 op/s 2024-05-17T04:50:22.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:22 smithi050 ceph-mon[31739]: pgmap v860: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.9 MiB/s wr, 324 op/s 2024-05-17T04:50:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 685.0 s, 94.6 tps, lat 1068.231 ms stddev 258.984 2024-05-17T04:50:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:24 smithi105 ceph-mon[34795]: pgmap v861: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-05-17T04:50:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:24 smithi110 ceph-mon[35497]: pgmap v861: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-05-17T04:50:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:24 smithi050 ceph-mon[31739]: pgmap v861: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-05-17T04:50:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:26 smithi105 ceph-mon[34795]: pgmap v862: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 347 op/s 2024-05-17T04:50:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:26 smithi110 ceph-mon[35497]: pgmap v862: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 347 op/s 2024-05-17T04:50:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:26 smithi050 ceph-mon[31739]: pgmap v862: 129 pgs: 129 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.3 MiB/s wr, 347 op/s 2024-05-17T04:50:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 690.0 s, 104.4 tps, lat 975.670 ms stddev 239.429 2024-05-17T04:50:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:28 smithi105 ceph-mon[34795]: pgmap v863: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-17T04:50:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:28 smithi110 ceph-mon[35497]: pgmap v863: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-17T04:50:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:28 smithi050 ceph-mon[31739]: pgmap v863: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.8 MiB/s wr, 325 op/s 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 -- 172.21.15.50:0/3211246097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 msgr2=0x7f86d81060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 --2- 172.21.15.50:0/3211246097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81060a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009900 tx=0x7f86d402a410 comp rx=0 tx=0).stop 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 -- 172.21.15.50:0/3211246097 shutdown_connections 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 --2- 172.21.15.50:0/3211246097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d8108f40 0x7f86d810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 --2- 172.21.15.50:0/3211246097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d81065e0 0x7f86d8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 --2- 172.21.15.50:0/3211246097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81060a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 -- 172.21.15.50:0/3211246097 >> 172.21.15.50:0/3211246097 conn(0x7f86d80fd8b0 msgr2=0x7f86d80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 -- 172.21.15.50:0/3211246097 shutdown_connections 2024-05-17T04:50:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.226+0000 7f86e0362640 1 -- 172.21.15.50:0/3211246097 wait complete. 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 Processor -- start 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 -- start start 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d81065e0 0x7f86d8101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 0x7f86d81020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810eb60 con 0x7f86d8103c80 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810e9e0 con 0x7f86d8108f40 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86e0362640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810ece0 con 0x7f86d81065e0 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86de0fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86de0fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46064/0 (socket says 172.21.15.50:46064) 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86de0fe640 1 -- 172.21.15.50:0/986917741 learned_addr learned my addr 172.21.15.50:0/986917741 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.227+0000 7f86de8ff640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 0x7f86d81020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d81065e0 0x7f86d8101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 -- 172.21.15.50:0/986917741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d81065e0 msgr2=0x7f86d8101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d81065e0 0x7f86d8101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 -- 172.21.15.50:0/986917741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 msgr2=0x7f86d81020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 0x7f86d81020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 -- 172.21.15.50:0/986917741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d8102800 con 0x7f86d8103c80 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de8ff640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 0x7f86d81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86de0fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f86d4006010 tx=0x7f86d4032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86cf7fe640 1 -- 172.21.15.50:0/986917741 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d403f030 con 0x7f86d8103c80 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.228+0000 7f86cf7fe640 1 -- 172.21.15.50:0/986917741 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d4003a80 con 0x7f86d8103c80 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.229+0000 7f86cf7fe640 1 -- 172.21.15.50:0/986917741 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4035070 con 0x7f86d8103c80 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.229+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d81a3890 con 0x7f86d8103c80 2024-05-17T04:50:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.229+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d81a3c40 con 0x7f86d8103c80 2024-05-17T04:50:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.232+0000 7f86cf7fe640 1 -- 172.21.15.50:0/986917741 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86d4042030 con 0x7f86d8103c80 2024-05-17T04:50:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 Processor -- start 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 -- start start 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c1fa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56225c15ccc0 con 0x56225be1e2f0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56225c14c860 con 0x56225be471b0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.233+0000 7f86e7218740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56225c1508a0 con 0x56225be264d0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46262/0 (socket says 172.21.15.50:46262) 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 -- 172.21.15.50:0/74586790 learned_addr learned my addr 172.21.15.50:0/74586790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de0fe640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c1fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 -- 172.21.15.50:0/74586790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 msgr2=0x56225c1fa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c1fa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 -- 172.21.15.50:0/74586790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 msgr2=0x56225c1f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 -- 172.21.15.50:0/74586790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56225c114d50 con 0x56225be471b0 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de0fe640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86cf7fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 0x7f86c40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86cf7fe640 1 -- 172.21.15.50:0/986917741 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86d41029f0 con 0x7f86d8103c80 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de8ff640 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86d00117e0 tx=0x7f86d0011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a158dd1b1d1fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.234+0000 7f86de0fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 0x7f86c40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.235+0000 7f86ccbfe640 1 -- 172.21.15.50:0/74586790 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0011310 con 0x56225be471b0 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.235+0000 7f86ccbfe640 1 -- 172.21.15.50:0/74586790 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d0003e00 con 0x56225be471b0 2024-05-17T04:50:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.235+0000 7f86ccbfe640 1 -- 172.21.15.50:0/74586790 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0004450 con 0x56225be471b0 2024-05-17T04:50:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- 172.21.15.50:0/74586790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 msgr2=0x56225c1f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86d00117e0 tx=0x7f86d0011ca0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86de0fe640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 0x7f86c40b4cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x56225c1f4380 tx=0x7f86d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- 172.21.15.50:0/74586790 shutdown_connections 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c1fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 --2- 172.21.15.50:0/74586790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c1f2f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- 172.21.15.50:0/74586790 >> 172.21.15.50:0/74586790 conn(0x56225be3b300 msgr2=0x56225c1e61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- 172.21.15.50:0/74586790 shutdown_connections 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- 172.21.15.50:0/74586790 wait complete. 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.236+0000 7f86e7218740 1 -- start start 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 Processor -- start 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c1fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be471b0 0x56225c203ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56225c088ab0 con 0x56225be1e2f0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56225c1094d0 con 0x56225be264d0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46082/0 (socket says 172.21.15.50:46082) 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56225c1faea0 con 0x56225be471b0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86de8ff640 1 -- 172.21.15.50:0/4200072896 learned_addr learned my addr 172.21.15.50:0/4200072896 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86e7218740 10 client.0 _ll_register_callbacks cb 0x56225c147950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.237+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c1fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be471b0 0x56225c203ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 -- 172.21.15.50:0/4200072896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be471b0 msgr2=0x56225c203ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be471b0 0x56225c203ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 -- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 msgr2=0x56225c1fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c1fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 -- 172.21.15.50:0/4200072896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56225c10a340 con 0x56225be1e2f0 2024-05-17T04:50:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c1fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86de8ff640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f86d0003c40 tx=0x7f86d00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b65b66617c3814 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d001e020 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86d0004c40 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d0003e00 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f86d0031050 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86e7218740 10 client.16410 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.238+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56225beb2790 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86d0003e00 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f86d0018020 con 0x56225be1e2f0 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 10 client.16410 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 20 client.16410 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225be05d80 0x56225c209e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 4 client.16410 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56225c15ec10 con 0x56225be05d80 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.239+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225be05d80 0x56225c209e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.240+0000 7f86c1ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.240+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225be05d80 0x56225c209e90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a247ad1fc18e server_cookie=10885ca00ed0f8dc in_seq=0 out_seq=0 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86c1ffb640 1 -- 172.21.15.50:0/4200072896 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56225c15ec10 con 0x56225be05d80 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86c1ffb640 10 client.16410 handle_command_reply: tid=0 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 client.16410 shutdown 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 msgr2=0x56225c1fc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f86d0003c40 tx=0x7f86d00039e0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 shutdown_connections 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be471b0 0x56225c203ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c1fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225be05d80 0x56225c209e90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 --2- 172.21.15.50:0/4200072896 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1fc840 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86c1ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 >> 172.21.15.50:0/4200072896 conn(0x56225be3b300 msgr2=0x56225c1e9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 shutdown_connections 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.241+0000 7f86e7218740 1 -- 172.21.15.50:0/4200072896 wait complete. 2024-05-17T04:50:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.242+0000 7f86e7218740 20 client.16410 trim_cache size 0 max 16384 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 Processor -- start 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 -- start start 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c204170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81ac620 con 0x56225be1e2f0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81a9ef0 con 0x56225be471b0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86e7218740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81a7d40 con 0x56225be264d0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86dd8fd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86dd8fd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56958/0 (socket says 172.21.15.50:56958) 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86dd8fd640 1 -- 172.21.15.50:0/3297394642 learned_addr learned my addr 172.21.15.50:0/3297394642 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:46086/0 (socket says 172.21.15.50:46086) 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.263+0000 7f86de8ff640 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c204170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 -- 172.21.15.50:0/3297394642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 msgr2=0x56225c204170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c204170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 -- 172.21.15.50:0/3297394642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 msgr2=0x56225c1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 -- 172.21.15.50:0/3297394642 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56225bf2b620 con 0x56225be264d0 2024-05-17T04:50:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56225c1e6e90 tx=0x7f86c8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d804d922ec9ca5b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86c3fff640 1 -- 172.21.15.50:0/3297394642 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8005020 con 0x56225be264d0 2024-05-17T04:50:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86c3fff640 1 -- 172.21.15.50:0/3297394642 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86c800f7f0 con 0x56225be264d0 2024-05-17T04:50:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.264+0000 7f86c3fff640 1 -- 172.21.15.50:0/3297394642 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004a00 con 0x56225be264d0 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- 172.21.15.50:0/3297394642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 msgr2=0x56225c203c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56225c1e6e90 tx=0x7f86c8005f90 comp rx=0 tx=0).stop 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- 172.21.15.50:0/3297394642 shutdown_connections 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be471b0 0x56225c204170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be264d0 0x56225c203c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 --2- 172.21.15.50:0/3297394642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be1e2f0 0x56225c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- 172.21.15.50:0/3297394642 >> 172.21.15.50:0/3297394642 conn(0x56225be3b300 msgr2=0x56225c1e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- 172.21.15.50:0/3297394642 shutdown_connections 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- 172.21.15.50:0/3297394642 wait complete. 2024-05-17T04:50:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.265+0000 7f86e7218740 1 -- start start 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 Processor -- start 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 0x56225c201250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be471b0 0x56225c2701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81a83b0 con 0x56225be471b0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81a9820 con 0x56225be264d0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86d81a8a80 con 0x56225be1e2f0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86e7218740 10 client.0 _ll_register_callbacks cb 0x7f86c80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 0x56225c201250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46280/0 (socket says 172.21.15.50:46280) 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86de8ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 0x56225c201250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56974/0 (socket says 172.21.15.50:56974) 2024-05-17T04:50:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 -- 172.21.15.50:0/631600144 learned_addr learned my addr 172.21.15.50:0/631600144 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86de8ff640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be471b0 0x56225c2701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 -- 172.21.15.50:0/631600144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 msgr2=0x56225c201250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 0x56225c201250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 -- 172.21.15.50:0/631600144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be471b0 msgr2=0x56225c2701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be471b0 0x56225c2701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.266+0000 7f86dd8fd640 1 -- 172.21.15.50:0/631600144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56225c204780 con 0x56225be264d0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86dd8fd640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86c8025040 tx=0x7f86c8012690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24123c95c283f60 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8032030 con 0x56225be264d0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c800f7f0 con 0x56225be264d0 2024-05-17T04:50:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86c8005230 con 0x56225be264d0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f86c803c050 con 0x56225be264d0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86e7218740 10 client.26165 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56225c271720 con 0x56225be264d0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86c800f7f0 con 0x56225be264d0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.267+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f86c8004a60 con 0x56225be264d0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 10 client.26165 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 20 client.26165 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225bddb1c0 0x56225c274140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 4 client.26165 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56225c1583f0 con 0x56225bddb1c0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86de8ff640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225bddb1c0 0x56225c274140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86c1ffb640 10 client.26165 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.268+0000 7f86de8ff640 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225bddb1c0 0x56225c274140 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc527292a07a2c73 server_cookie=d50bd8c6255d6641 in_seq=0 out_seq=0 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86c1ffb640 1 -- 172.21.15.50:0/631600144 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56225c1583f0 con 0x56225bddb1c0 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86c1ffb640 10 client.26165 handle_command_reply: tid=0 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 client.26165 shutdown 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 msgr2=0x56225c201790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f86c8025040 tx=0x7f86c8012690 comp rx=0 tx=0).stop 2024-05-17T04:50:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 shutdown_connections 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56225bddb1c0 0x56225c274140 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56225be471b0 0x56225c2701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56225be264d0 0x56225c201790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86c1ffb640 0 client.26165 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 --2- 172.21.15.50:0/631600144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56225be1e2f0 0x56225c201250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 >> 172.21.15.50:0/631600144 conn(0x56225be3b300 msgr2=0x56225c1e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 shutdown_connections 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.294+0000 7f86e7218740 1 -- 172.21.15.50:0/631600144 wait complete. 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.295+0000 7f86e7218740 20 client.26165 trim_cache size 0 max 16384 2024-05-17T04:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:50:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:50:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.295+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 msgr2=0x7f86c40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.295+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 0x7f86c40b4cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x56225c1f4380 tx=0x7f86d4009260 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.295+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 msgr2=0x7f86d81034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.295+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f86d4006010 tx=0x7f86d4032b40 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 shutdown_connections 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86d8108f40 0x7f86d81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86d81065e0 0x7f86d8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86c40b27e0 0x7f86c40b4cf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 --2- 172.21.15.50:0/986917741 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86d8103c80 0x7f86d81034f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 >> 172.21.15.50:0/986917741 conn(0x7f86d80fd8b0 msgr2=0x7f86d8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 shutdown_connections 2024-05-17T04:50:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:29.296+0000 7f86e0362640 1 -- 172.21.15.50:0/986917741 wait complete. 2024-05-17T04:50:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:30 smithi105 ceph-mon[34795]: pgmap v864: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-05-17T04:50:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:30 smithi110 ceph-mon[35497]: pgmap v864: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-05-17T04:50:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:30 smithi050 ceph-mon[31739]: pgmap v864: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 1.7 MiB/s wr, 301 op/s 2024-05-17T04:50:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:32 smithi105 ceph-mon[34795]: pgmap v865: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-05-17T04:50:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:32 smithi110 ceph-mon[35497]: pgmap v865: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-05-17T04:50:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:32 smithi050 ceph-mon[31739]: pgmap v865: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-05-17T04:50:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 695.0 s, 112.2 tps, lat 883.771 ms stddev 285.428 2024-05-17T04:50:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:34 smithi105 ceph-mon[34795]: pgmap v866: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.2 MiB/s wr, 363 op/s 2024-05-17T04:50:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:34 smithi110 ceph-mon[35497]: pgmap v866: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.2 MiB/s wr, 363 op/s 2024-05-17T04:50:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:34 smithi050 ceph-mon[31739]: pgmap v866: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 2.2 MiB/s wr, 363 op/s 2024-05-17T04:50:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:36 smithi105 ceph-mon[34795]: pgmap v867: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-05-17T04:50:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:36 smithi110 ceph-mon[35497]: pgmap v867: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-05-17T04:50:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:36 smithi050 ceph-mon[31739]: pgmap v867: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.8 MiB/s wr, 309 op/s 2024-05-17T04:50:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 700.0 s, 100.2 tps, lat 1004.693 ms stddev 226.236 2024-05-17T04:50:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:38 smithi105 ceph-mon[34795]: pgmap v868: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-05-17T04:50:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:38 smithi110 ceph-mon[35497]: pgmap v868: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-05-17T04:50:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:38 smithi050 ceph-mon[31739]: pgmap v868: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-05-17T04:50:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:40 smithi105 ceph-mon[34795]: pgmap v869: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-05-17T04:50:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:40 smithi110 ceph-mon[35497]: pgmap v869: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-05-17T04:50:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:40 smithi050 ceph-mon[31739]: pgmap v869: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-05-17T04:50:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 -- 172.21.15.50:0/3307710743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8678108f30 msgr2=0x7f867810b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 --2- 172.21.15.50:0/3307710743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8678108f30 0x7f867810b350 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8670009ca0 tx=0x7f867002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 -- 172.21.15.50:0/3307710743 shutdown_connections 2024-05-17T04:50:42.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 --2- 172.21.15.50:0/3307710743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8678108f30 0x7f867810b350 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 --2- 172.21.15.50:0/3307710743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86781065d0 0x7f86781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 --2- 172.21.15.50:0/3307710743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678103c70 0x7f8678106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.437+0000 7f867cc8f640 1 -- 172.21.15.50:0/3307710743 >> 172.21.15.50:0/3307710743 conn(0x7f86780fd880 msgr2=0x7f86780ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.438+0000 7f867cc8f640 1 -- 172.21.15.50:0/3307710743 shutdown_connections 2024-05-17T04:50:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.438+0000 7f867cc8f640 1 -- 172.21.15.50:0/3307710743 wait complete. 2024-05-17T04:50:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 Processor -- start 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 -- start start 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 0x7f86781034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678108f30 0x7f86781020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f867810eba0 con 0x7f86781065d0 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f867810ea20 con 0x7f8678103c70 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867cc8f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f867810ed20 con 0x7f8678108f30 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f866ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f867659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 0x7f86781034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f866ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35252/0 (socket says 172.21.15.50:35252) 2024-05-17T04:50:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f866ffff640 1 -- 172.21.15.50:0/1390807424 learned_addr learned my addr 172.21.15.50:0/1390807424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.439+0000 7f8676d9d640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678108f30 0x7f86781020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 -- 172.21.15.50:0/1390807424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678108f30 msgr2=0x7f86781020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678108f30 0x7f86781020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 -- 172.21.15.50:0/1390807424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 msgr2=0x7f86781034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 0x7f86781034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 -- 172.21.15.50:0/1390807424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867819d620 con 0x7f86781065d0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f867659c640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 0x7f86781034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866ffff640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f866000e9f0 tx=0x7f866000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866f7fe640 1 -- 172.21.15.50:0/1390807424 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866000cd80 con 0x7f86781065d0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866f7fe640 1 -- 172.21.15.50:0/1390807424 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86600108c0 con 0x7f86781065d0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8678102720 con 0x7f86781065d0 2024-05-17T04:50:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f866f7fe640 1 -- 172.21.15.50:0/1390807424 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86600187c0 con 0x7f86781065d0 2024-05-17T04:50:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.440+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781a8220 con 0x7f86781065d0 2024-05-17T04:50:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 Processor -- start 2024-05-17T04:50:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 -- start start 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562a4662dcc0 con 0x562a462f74d0 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a4661d860 con 0x562a462ef2f0 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.444+0000 7f8683c24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a466218a0 con 0x562a463181b0 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466bb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56454/0 (socket says 172.21.15.50:56454) 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f8676d9d640 1 -- 172.21.15.50:0/3431384615 learned_addr learned my addr 172.21.15.50:0/3431384615 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 -- 172.21.15.50:0/3431384615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 msgr2=0x562a466bb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466bb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 -- 172.21.15.50:0/3431384615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 msgr2=0x562a466c2c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c2c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 -- 172.21.15.50:0/3431384615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a465e5d50 con 0x562a462ef2f0 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f866f7fe640 1 -- 172.21.15.50:0/1390807424 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8660010430 con 0x7f86781065d0 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f867659c640 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466bbcf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8668009fa0 tx=0x7f86680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d108b12de32a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f866cff9640 1 -- 172.21.15.50:0/3431384615 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668004030 con 0x562a462ef2f0 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.445+0000 7f866cff9640 1 -- 172.21.15.50:0/3431384615 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86680035f0 con 0x562a462ef2f0 2024-05-17T04:50:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.446+0000 7f866cff9640 1 -- 172.21.15.50:0/3431384615 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800f7c0 con 0x562a462ef2f0 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f866f7fe640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 0x7f86640b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8676d9d640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 0x7f86640b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f866f7fe640 1 -- 172.21.15.50:0/1390807424 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86600e2870 con 0x7f86781065d0 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 -- 172.21.15.50:0/3431384615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 msgr2=0x562a466bbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466bbcf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8668009fa0 tx=0x7f86680118d0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 -- 172.21.15.50:0/3431384615 shutdown_connections 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466bbcf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 --2- 172.21.15.50:0/3431384615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466bb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 -- 172.21.15.50:0/3431384615 >> 172.21.15.50:0/3431384615 conn(0x562a4630c300 msgr2=0x562a466b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.447+0000 7f8683c24740 1 -- 172.21.15.50:0/3431384615 shutdown_connections 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.448+0000 7f8683c24740 1 -- 172.21.15.50:0/3431384615 wait complete. 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.448+0000 7f8676d9d640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 0x7f86640b4d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86700094e0 tx=0x7f8670031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.448+0000 7f8683c24740 1 -- start start 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.448+0000 7f8683c24740 1 Processor -- start 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f867659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f867659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:48002/0 (socket says 172.21.15.50:48002) 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f866ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f866ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35262/0 (socket says 172.21.15.50:35262) 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f867659c640 1 -- 172.21.15.50:0/2846144000 learned_addr learned my addr 172.21.15.50:0/2846144000 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8676d9d640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562a46559ab0 con 0x562a462f74d0 2024-05-17T04:50:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a465da4d0 con 0x562a462ef2f0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a466c3690 con 0x562a463181b0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.449+0000 7f8683c24740 10 client.0 _ll_register_callbacks cb 0x562a46618950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 -- 172.21.15.50:0/2846144000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 msgr2=0x562a466cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 -- 172.21.15.50:0/2846144000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 msgr2=0x562a466c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a465db340 con 0x562a462ef2f0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f867659c640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f866801b040 tx=0x7f8668004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bb7bfa88a3204e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f865dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668028020 con 0x562a462ef2f0 2024-05-17T04:50:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8668016040 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.450+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86680035f0 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8668033050 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 10 client.26177 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562a46383790 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86680035f0 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f866800f5a0 con 0x562a462ef2f0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 10 client.26177 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 20 client.26177 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.451+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462d6d80 0x562a466d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.452+0000 7f8683c24740 4 client.26177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.452+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4662fc10 con 0x562a462d6d80 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.452+0000 7f866ffff640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462d6d80 0x562a466d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.452+0000 7f865dffb640 10 client.26177 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.452+0000 7f866ffff640 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462d6d80 0x562a466d2740 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84fb7a53207e2a7 server_cookie=6cac14d37868fe0 in_seq=0 out_seq=0 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.453+0000 7f865dffb640 1 -- 172.21.15.50:0/2846144000 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a4662fc10 con 0x562a462d6d80 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.453+0000 7f865dffb640 10 client.26177 handle_command_reply: tid=0 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.453+0000 7f8683c24740 1 client.26177 shutdown 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 msgr2=0x562a466c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f866801b040 tx=0x7f8668004600 comp rx=0 tx=0).stop 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 shutdown_connections 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a463181b0 0x562a466cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462d6d80 0x562a466d2740 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 --2- 172.21.15.50:0/2846144000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a462ef2f0 0x562a466c5120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f865dffb640 0 client.26177 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 >> 172.21.15.50:0/2846144000 conn(0x562a4630c300 msgr2=0x562a466ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 shutdown_connections 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.454+0000 7f8683c24740 1 -- 172.21.15.50:0/2846144000 wait complete. 2024-05-17T04:50:42.460 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.455+0000 7f8683c24740 20 client.26177 trim_cache size 0 max 16384 2024-05-17T04:50:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.479+0000 7f8683c24740 1 Processor -- start 2024-05-17T04:50:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.479+0000 7f8683c24740 1 -- start start 2024-05-17T04:50:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.479+0000 7f8683c24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 0x562a466c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8683c24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462f74d0 0x562a466cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8683c24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8683c24740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b4ec0 con 0x562a462ef2f0 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8683c24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b2790 con 0x562a463181b0 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8683c24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b05e0 con 0x562a462f74d0 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:48006/0 (socket says 172.21.15.50:48006) 2024-05-17T04:50:42.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f8676d9d640 1 -- 172.21.15.50:0/196895224 learned_addr learned my addr 172.21.15.50:0/196895224 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.480+0000 7f867659c640 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 0x562a466c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 -- 172.21.15.50:0/196895224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462f74d0 msgr2=0x562a466cc4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462f74d0 0x562a466cc4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 -- 172.21.15.50:0/196895224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 msgr2=0x562a466c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 0x562a466c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 -- 172.21.15.50:0/196895224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a463fc620 con 0x562a463181b0 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f867659c640 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 0x562a466c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:42.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f8676d9d640 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8670035000 tx=0x7f8670044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5a5a4b0e563f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f865ffff640 1 -- 172.21.15.50:0/196895224 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670003b70 con 0x562a463181b0 2024-05-17T04:50:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f865ffff640 1 -- 172.21.15.50:0/196895224 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8670036050 con 0x562a463181b0 2024-05-17T04:50:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.481+0000 7f865ffff640 1 -- 172.21.15.50:0/196895224 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86700308b0 con 0x562a463181b0 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.482+0000 7f8683c24740 1 -- 172.21.15.50:0/196895224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 msgr2=0x562a466c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8670035000 tx=0x7f8670044000 comp rx=0 tx=0).stop 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 -- 172.21.15.50:0/196895224 shutdown_connections 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a466c4c20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462f74d0 0x562a466cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 --2- 172.21.15.50:0/196895224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462ef2f0 0x562a466c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 -- 172.21.15.50:0/196895224 >> 172.21.15.50:0/196895224 conn(0x562a4630c300 msgr2=0x562a466b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 -- 172.21.15.50:0/196895224 shutdown_connections 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 -- 172.21.15.50:0/196895224 wait complete. 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.483+0000 7f8683c24740 1 -- start start 2024-05-17T04:50:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f8683c24740 1 Processor -- start 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f8683c24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f8683c24740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f8683c24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a46738aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f8683c24740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b0c50 con 0x562a462f74d0 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f867659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f866ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f867659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:56496/0 (socket says 172.21.15.50:56496) 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.484+0000 7f867659c640 1 -- 172.21.15.50:0/2939336773 learned_addr learned my addr 172.21.15.50:0/2939336773 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f8676d9d640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a46738aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b20c0 con 0x562a463181b0 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86781b1320 con 0x562a462ef2f0 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f8683c24740 10 client.0 _ll_register_callbacks cb 0x7f8668005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 -- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 msgr2=0x562a46738aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a46738aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 -- 172.21.15.50:0/2939336773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 msgr2=0x562a466c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a46738fe0 con 0x562a462ef2f0 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f8676d9d640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a46738aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f867659c640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f866801b040 tx=0x7f8668009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16cac9265bca3a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.485+0000 7f865dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866803c060 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8668003b20 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8670036050 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8668043050 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f8683c24740 10 client.26155 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562a4673a1b0 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.486+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8670036050 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f8668004680 con 0x562a462ef2f0 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 10 client.26155 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 20 client.26155 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462ac1c0 0x562a4673ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 4 client.26155 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a466293f0 con 0x562a462ac1c0 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.487+0000 7f866ffff640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462ac1c0 0x562a4673ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.488+0000 7f865dffb640 10 client.26155 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.488+0000 7f866ffff640 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462ac1c0 0x562a4673ccb0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2866b1363a9b90b server_cookie=3f463cb63b5fb475 in_seq=0 out_seq=0 2024-05-17T04:50:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f865dffb640 1 -- 172.21.15.50:0/2939336773 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a466293f0 con 0x562a462ac1c0 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f865dffb640 10 client.26155 handle_command_reply: tid=0 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 client.26155 shutdown 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 msgr2=0x562a466c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f866801b040 tx=0x7f8668009e90 comp rx=0 tx=0).stop 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 shutdown_connections 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a463181b0 0x562a46738aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562a462ac1c0 0x562a4673ccb0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562a462f74d0 0x562a466c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 --2- 172.21.15.50:0/2939336773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a462ef2f0 0x562a466c94d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f865dffb640 0 client.26155 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 >> 172.21.15.50:0/2939336773 conn(0x562a4630c300 msgr2=0x562a466ba790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 shutdown_connections 2024-05-17T04:50:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 1 -- 172.21.15.50:0/2939336773 wait complete. 2024-05-17T04:50:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.514+0000 7f8683c24740 20 client.26155 trim_cache size 0 max 16384 2024-05-17T04:50:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:50:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:50:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 msgr2=0x7f86640b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 0x7f86640b4d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86700094e0 tx=0x7f8670031000 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 msgr2=0x7f8678101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f866000e9f0 tx=0x7f866000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 shutdown_connections 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8678108f30 0x7f86781020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f86640b2860 0x7f86640b4d70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86781065d0 0x7f8678101b80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 --2- 172.21.15.50:0/1390807424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8678103c70 0x7f86781034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 >> 172.21.15.50:0/1390807424 conn(0x7f86780fd880 msgr2=0x7f8678104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 shutdown_connections 2024-05-17T04:50:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:42.515+0000 7f867cc8f640 1 -- 172.21.15.50:0/1390807424 wait complete. 2024-05-17T04:50:42.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:42 smithi105 ceph-mon[34795]: pgmap v870: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.4 MiB/s wr, 362 op/s 2024-05-17T04:50:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:42 smithi110 ceph-mon[35497]: pgmap v870: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.4 MiB/s wr, 362 op/s 2024-05-17T04:50:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:42 smithi050 ceph-mon[31739]: pgmap v870: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.4 MiB/s wr, 362 op/s 2024-05-17T04:50:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 705.0 s, 100.2 tps, lat 997.230 ms stddev 229.714 2024-05-17T04:50:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:44 smithi105 ceph-mon[34795]: pgmap v871: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.0 MiB/s wr, 355 op/s 2024-05-17T04:50:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:44 smithi110 ceph-mon[35497]: pgmap v871: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.0 MiB/s wr, 355 op/s 2024-05-17T04:50:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:44 smithi050 ceph-mon[31739]: pgmap v871: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 2.0 MiB/s wr, 355 op/s 2024-05-17T04:50:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:46 smithi105 ceph-mon[34795]: pgmap v872: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-05-17T04:50:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:46 smithi110 ceph-mon[35497]: pgmap v872: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-05-17T04:50:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:46 smithi050 ceph-mon[31739]: pgmap v872: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.8 MiB/s wr, 312 op/s 2024-05-17T04:50:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 710.0 s, 97.0 tps, lat 1027.046 ms stddev 243.109 2024-05-17T04:50:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:48 smithi105 ceph-mon[34795]: pgmap v873: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:50:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:48 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:50:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:48 smithi110 ceph-mon[35497]: pgmap v873: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:50:48.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:48 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:50:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:48 smithi050 ceph-mon[31739]: pgmap v873: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-05-17T04:50:48.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:48 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:50:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:50:49.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:50:49.567 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:50:49.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:50:49.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi105 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:50:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:50:49.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:50:49.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:50:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:50 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:50:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:50 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:50:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:50 smithi105 ceph-mon[34795]: pgmap v874: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-17T04:50:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:50 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:50:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:50 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:50:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:50 smithi110 ceph-mon[35497]: pgmap v874: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-17T04:50:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:50 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-17T04:50:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:50 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi050 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-17T04:50:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:50 smithi050 ceph-mon[31739]: pgmap v874: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 305 op/s 2024-05-17T04:50:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:52 smithi105 ceph-mon[34795]: pgmap v875: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:52 smithi110 ceph-mon[35497]: pgmap v875: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:52 smithi050 ceph-mon[31739]: pgmap v875: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 715.0 s, 100.4 tps, lat 1006.827 ms stddev 243.136 2024-05-17T04:50:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:54 smithi105 ceph-mon[34795]: pgmap v876: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-05-17T04:50:54.569 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:54 smithi110 ceph-mon[35497]: pgmap v876: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-05-17T04:50:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:54 smithi050 ceph-mon[31739]: pgmap v876: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.1 MiB/s wr, 344 op/s 2024-05-17T04:50:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/3685912729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48103280 msgr2=0x7f3f48109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/3685912729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48103280 0x7f3f48109b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009ca0 tx=0x7f3f4002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/3685912729 shutdown_connections 2024-05-17T04:50:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/3685912729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48103280 0x7f3f48109b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/3685912729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48102930 0x7f3f48102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/3685912729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f48101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/3685912729 >> 172.21.15.50:0/3685912729 conn(0x7f3f480fd030 msgr2=0x7f3f480ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.646+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/3685912729 shutdown_connections 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.647+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/3685912729 wait complete. 2024-05-17T04:50:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.647+0000 7f3f4e8f6640 1 Processor -- start 2024-05-17T04:50:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.647+0000 7f3f4e8f6640 1 -- start start 2024-05-17T04:50:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102930 0x7f3f4819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48103280 0x7f3f481a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d1a0 con 0x7f3f48103280 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d020 con 0x7f3f48102930 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4e8f6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d320 con 0x7f3f48101810 2024-05-17T04:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102930 0x7f3f4819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:54008/0 (socket says 172.21.15.50:54008) 2024-05-17T04:50:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 -- 172.21.15.50:0/4252384820 learned_addr learned my addr 172.21.15.50:0/4252384820 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48103280 0x7f3f481a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 -- 172.21.15.50:0/4252384820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102930 msgr2=0x7f3f4819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102930 0x7f3f4819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 -- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48103280 msgr2=0x7f3f481a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48103280 0x7f3f481a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.648+0000 7f3f47fff640 1 -- 172.21.15.50:0/4252384820 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f481a10f0 con 0x7f3f48101810 2024-05-17T04:50:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f47fff640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800e9f0 tx=0x7f3f3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f457fa640 1 -- 172.21.15.50:0/4252384820 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800ccc0 con 0x7f3f48101810 2024-05-17T04:50:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f481a13e0 con 0x7f3f48101810 2024-05-17T04:50:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f457fa640 1 -- 172.21.15.50:0/4252384820 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f380054b0 con 0x7f3f48101810 2024-05-17T04:50:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f457fa640 1 -- 172.21.15.50:0/4252384820 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800ccc0 con 0x7f3f48101810 2024-05-17T04:50:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.649+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f481a18e0 con 0x7f3f48101810 2024-05-17T04:50:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.652+0000 7f3f457fa640 1 -- 172.21.15.50:0/4252384820 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f380046c0 con 0x7f3f48101810 2024-05-17T04:50:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.652+0000 7f3f557b2740 1 Processor -- start 2024-05-17T04:50:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 -- start start 2024-05-17T04:50:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 0x557a4dd35770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f457fa640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 0x7f3f300b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9714d0 0x557a4dd3cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557a4dca7cc0 con 0x557a4d9921b0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a4dc97860 con 0x557a4d9714d0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f557b2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a4dc9b8a0 con 0x557a4d9692f0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f4ce93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 0x557a4dd35770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:54010/0 (socket says 172.21.15.50:54010) 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f477fe640 1 -- 172.21.15.50:0/3761937671 learned_addr learned my addr 172.21.15.50:0/3761937671 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9714d0 0x557a4dd3cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f457fa640 1 -- 172.21.15.50:0/4252384820 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3f380e2520 con 0x7f3f48101810 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.653+0000 7f3f47fff640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 0x7f3f300b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 -- 172.21.15.50:0/3761937671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 msgr2=0x557a4dd35cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 -- 172.21.15.50:0/3761937671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9714d0 msgr2=0x557a4dd3cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9714d0 0x557a4dd3cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 -- 172.21.15.50:0/3761937671 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4dc5fd50 con 0x557a4d9921b0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f477fe640 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f47fff640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 0x7f3f300b4ee0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3f380a6a20 tx=0x7f3f380049b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 0x557a4dd35770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f4002ade0 tx=0x7f3f4002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3af56b98a13b9ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f2e7fc640 1 -- 172.21.15.50:0/3761937671 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40031640 con 0x557a4d9921b0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f2e7fc640 1 -- 172.21.15.50:0/3761937671 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f40036070 con 0x557a4d9921b0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.654+0000 7f3f2e7fc640 1 -- 172.21.15.50:0/3761937671 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40004030 con 0x557a4d9921b0 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- 172.21.15.50:0/3761937671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 msgr2=0x557a4dd35770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 0x557a4dd35770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f4002ade0 tx=0x7f3f4002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- 172.21.15.50:0/3761937671 shutdown_connections 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9714d0 0x557a4dd3cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd35cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3761937671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9921b0 0x557a4dd35770 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- 172.21.15.50:0/3761937671 >> 172.21.15.50:0/3761937671 conn(0x557a4d986300 msgr2=0x557a4dd311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- 172.21.15.50:0/3761937671 shutdown_connections 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- 172.21.15.50:0/3761937671 wait complete. 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.656+0000 7f3f557b2740 1 -- start start 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f557b2740 1 Processor -- start 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 0x557a4dd3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 0x557a4dd46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f4ce93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 0x557a4dd3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f4ce93640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 0x557a4dd3eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:44766/0 (socket says 172.21.15.50:44766) 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:54024/0 (socket says 172.21.15.50:54024) 2024-05-17T04:50:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.657+0000 7f3f4ce93640 1 -- 172.21.15.50:0/2042614771 learned_addr learned my addr 172.21.15.50:0/2042614771 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557a4dbd3ab0 con 0x557a4d9692f0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a4dc544d0 con 0x557a4d9921b0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a4dd3d4b0 con 0x557a4d9714d0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f557b2740 10 client.0 _ll_register_callbacks cb 0x557a4dc92950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 0x557a4dd46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f477fe640 1 -- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 msgr2=0x557a4dd46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 0x557a4dd46360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f477fe640 1 -- 172.21.15.50:0/2042614771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 msgr2=0x557a4dd3eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 0x557a4dd3eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f477fe640 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4dc55340 con 0x557a4d9714d0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.658+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 0x557a4dd46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557a4dd36b70 tx=0x7f3f3c012b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980837c926eea43d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f277fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:50:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f40036070 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f277fe640 1 -- 172.21.15.50:0/2042614771 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c004050 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f277fe640 1 -- 172.21.15.50:0/2042614771 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3f3c0041f0 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f277fe640 1 -- 172.21.15.50:0/2042614771 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3f3c004430 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 10 client.26167 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=41}) -- 0x557a4d9fd790 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f277fe640 1 -- 172.21.15.50:0/2042614771 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f3f3c00f8f0 con 0x557a4d9714d0 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 10 client.26167 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.659+0000 7f3f557b2740 20 client.26167 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d950d80 0x557a4dd4c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f557b2740 4 client.26167 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4dca9c10 con 0x557a4d950d80 2024-05-17T04:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d950d80 0x557a4dd4c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f277fe640 10 client.26167 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.660+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d950d80 0x557a4dd4c610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ef793ec449e02 server_cookie=15e76589bdd95003 in_seq=0 out_seq=0 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f277fe640 1 -- 172.21.15.50:0/2042614771 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a4dca9c10 con 0x557a4d950d80 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f277fe640 10 client.26167 handle_command_reply: tid=0 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 client.26167 shutdown 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 msgr2=0x557a4dd3f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557a4dd36b70 tx=0x7f3f3c012b40 comp rx=0 tx=0).stop 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 shutdown_connections 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9921b0 0x557a4dd46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9714d0 0x557a4dd3f400 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d950d80 0x557a4dd4c610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2042614771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9692f0 0x557a4dd3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f277fe640 0 client.26167 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 >> 172.21.15.50:0/2042614771 conn(0x557a4d986300 msgr2=0x557a4dd34010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 shutdown_connections 2024-05-17T04:50:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 1 -- 172.21.15.50:0/2042614771 wait complete. 2024-05-17T04:50:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.661+0000 7f3f557b2740 20 client.26167 trim_cache size 0 max 16384 2024-05-17T04:50:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.686+0000 7f3f557b2740 1 Processor -- start 2024-05-17T04:50:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.686+0000 7f3f557b2740 1 -- start start 2024-05-17T04:50:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d950d80 0x557a4dd39bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4dd46280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481ae7e0 con 0x557a4d9714d0 2024-05-17T04:50:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481ac0b0 con 0x557a4d9692f0 2024-05-17T04:50:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f557b2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481a9f00 con 0x557a4d950d80 2024-05-17T04:50:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40120/0 (socket says 172.21.15.50:40120) 2024-05-17T04:50:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 -- 172.21.15.50:0/3400142016 learned_addr learned my addr 172.21.15.50:0/3400142016 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d950d80 0x557a4dd39bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4dd46280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 -- 172.21.15.50:0/3400142016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d950d80 msgr2=0x557a4dd39bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d950d80 0x557a4dd39bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 -- 172.21.15.50:0/3400142016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 msgr2=0x557a4dd46280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4dd46280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f477fe640 1 -- 172.21.15.50:0/3400142016 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4da76620 con 0x557a4d9692f0 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.687+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4dd46280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:50:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.688+0000 7f3f477fe640 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009560 tx=0x7f3f3c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22081a99fbf524b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.688+0000 7f3f2dffb640 1 -- 172.21.15.50:0/3400142016 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00fe70 con 0x557a4d9692f0 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.688+0000 7f3f2dffb640 1 -- 172.21.15.50:0/3400142016 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f40036070 con 0x557a4d9692f0 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.688+0000 7f3f2dffb640 1 -- 172.21.15.50:0/3400142016 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c01c420 con 0x557a4d9692f0 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 -- 172.21.15.50:0/3400142016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 msgr2=0x557a4dd3da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009560 tx=0x7f3f3c013000 comp rx=0 tx=0).stop 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 -- 172.21.15.50:0/3400142016 shutdown_connections 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4dd46280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d9692f0 0x557a4dd3da80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 --2- 172.21.15.50:0/3400142016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d950d80 0x557a4dd39bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.689+0000 7f3f557b2740 1 -- 172.21.15.50:0/3400142016 >> 172.21.15.50:0/3400142016 conn(0x557a4d986300 msgr2=0x557a4dd33ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.690+0000 7f3f557b2740 1 -- 172.21.15.50:0/3400142016 shutdown_connections 2024-05-17T04:50:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.690+0000 7f3f557b2740 1 -- 172.21.15.50:0/3400142016 wait complete. 2024-05-17T04:50:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.690+0000 7f3f557b2740 1 -- start start 2024-05-17T04:50:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.691+0000 7f3f557b2740 1 Processor -- start 2024-05-17T04:50:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.691+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f557b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f4ce93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f4ce93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40132/0 (socket says 172.21.15.50:40132) 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f4ce93640 1 -- 172.21.15.50:0/2522155106 learned_addr learned my addr 172.21.15.50:0/2522155106 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4ddb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481aa570 con 0x557a4d9714d0 2024-05-17T04:50:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481ab9e0 con 0x557a4d950d80 2024-05-17T04:50:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481aac40 con 0x557a4d9692f0 2024-05-17T04:50:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.692+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4ddb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 -- 172.21.15.50:0/2522155106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 msgr2=0x557a4dd459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 -- 172.21.15.50:0/2522155106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 msgr2=0x557a4ddb2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:50:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4ddb2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4ddb30f0 con 0x557a4d950d80 2024-05-17T04:50:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f557b2740 10 client.0 _ll_register_callbacks cb 0x7f3f3c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:50:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f40036070 con 0x557a4d950d80 2024-05-17T04:50:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f4ce93640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009d90 tx=0x7f3f4004d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa53f74eb3d56418 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:50:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f277fe640 10 client.26201 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:50:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f277fe640 1 -- 172.21.15.50:0/2522155106 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40064060 con 0x557a4d950d80 2024-05-17T04:50:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f277fe640 1 -- 172.21.15.50:0/2522155106 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3f40031a80 con 0x557a4d950d80 2024-05-17T04:50:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f277fe640 1 -- 172.21.15.50:0/2522155106 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3f40031c80 con 0x557a4d950d80 2024-05-17T04:50:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f557b2740 10 client.26201 fetch_fsmap learned FSMap version 41 2024-05-17T04:50:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.693+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=41}) -- 0x557a4ddb4100 con 0x557a4d950d80 2024-05-17T04:50:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f277fe640 1 -- 172.21.15.50:0/2522155106 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f3f40058250 con 0x557a4d950d80 2024-05-17T04:50:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 10 client.26201 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:50:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:50:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 20 client.26201 populate_metadata read hostname 'smithi050' 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d9921b0 0x557a4ddb6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 4 client.26201 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a4dca33f0 con 0x557a4d9921b0 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.694+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d9921b0 0x557a4ddb6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.695+0000 7f3f477fe640 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d9921b0 0x557a4ddb6790 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49429adfc28274d9 server_cookie=5fec9af2cdb08c3c in_seq=0 out_seq=0 2024-05-17T04:50:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.695+0000 7f3f277fe640 10 client.26201 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f277fe640 1 -- 172.21.15.50:0/2522155106 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a4dca33f0 con 0x557a4d9921b0 2024-05-17T04:50:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f277fe640 10 client.26201 handle_command_reply: tid=0 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 client.26201 shutdown 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 msgr2=0x557a4dd454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009d90 tx=0x7f3f4004d040 comp rx=0 tx=0).stop 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 shutdown_connections 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557a4d9921b0 0x557a4ddb6790 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557a4d9714d0 0x557a4ddb2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a4d9692f0 0x557a4dd459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 --2- 172.21.15.50:0/2522155106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a4d950d80 0x557a4dd454a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 >> 172.21.15.50:0/2522155106 conn(0x557a4d986300 msgr2=0x557a4dd38bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f277fe640 0 client.26201 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 shutdown_connections 2024-05-17T04:50:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.716+0000 7f3f557b2740 1 -- 172.21.15.50:0/2522155106 wait complete. 2024-05-17T04:50:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f557b2740 20 client.26201 trim_cache size 0 max 16384 2024-05-17T04:50:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:50:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:50:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:50:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 msgr2=0x7f3f300b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 0x7f3f300b4ee0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3f380a6a20 tx=0x7f3f380049b0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 msgr2=0x7f3f480feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800e9f0 tx=0x7f3f3800eeb0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 shutdown_connections 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3f300b29d0 0x7f3f300b4ee0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f48103280 0x7f3f481a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102930 0x7f3f4819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 --2- 172.21.15.50:0/4252384820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f48101810 0x7f3f480feed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:50:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 >> 172.21.15.50:0/4252384820 conn(0x7f3f480fd030 msgr2=0x7f3f48106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:50:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.717+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 shutdown_connections 2024-05-17T04:50:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:50:55.718+0000 7f3f4e8f6640 1 -- 172.21.15.50:0/4252384820 wait complete. 2024-05-17T04:50:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:56 smithi105 ceph-mon[34795]: pgmap v877: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:56 smithi110 ceph-mon[35497]: pgmap v877: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:56 smithi050 ceph-mon[31739]: pgmap v877: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.9 MiB/s wr, 309 op/s 2024-05-17T04:50:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 720.0 s, 106.0 tps, lat 940.715 ms stddev 236.562 2024-05-17T04:50:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:50:58 smithi105 ceph-mon[34795]: pgmap v878: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:50:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:50:58 smithi110 ceph-mon[35497]: pgmap v878: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:50:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:50:58 smithi050 ceph-mon[31739]: pgmap v878: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.9 MiB/s wr, 322 op/s 2024-05-17T04:51:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:00 smithi105 ceph-mon[34795]: pgmap v879: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:00 smithi110 ceph-mon[35497]: pgmap v879: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:00 smithi050 ceph-mon[31739]: pgmap v879: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:02 smithi105 ceph-mon[34795]: pgmap v880: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-05-17T04:51:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:02 smithi110 ceph-mon[35497]: pgmap v880: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-05-17T04:51:02.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:02 smithi050 ceph-mon[31739]: pgmap v880: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-05-17T04:51:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 725.0 s, 115.4 tps, lat 866.332 ms stddev 314.296 2024-05-17T04:51:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:04 smithi105 ceph-mon[34795]: pgmap v881: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-05-17T04:51:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:04 smithi110 ceph-mon[35497]: pgmap v881: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-05-17T04:51:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:04 smithi050 ceph-mon[31739]: pgmap v881: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-05-17T04:51:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:06 smithi105 ceph-mon[34795]: pgmap v882: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-05-17T04:51:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:06 smithi110 ceph-mon[35497]: pgmap v882: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-05-17T04:51:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:06 smithi050 ceph-mon[31739]: pgmap v882: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.7 MiB/s wr, 311 op/s 2024-05-17T04:51:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 730.0 s, 97.8 tps, lat 1019.115 ms stddev 244.270 2024-05-17T04:51:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:08 smithi105 ceph-mon[34795]: pgmap v883: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:51:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:08 smithi110 ceph-mon[35497]: pgmap v883: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:51:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:08 smithi050 ceph-mon[31739]: pgmap v883: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:51:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 -- 172.21.15.50:0/4267186758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103000 msgr2=0x7f68d0103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 --2- 172.21.15.50:0/4267186758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103000 0x7f68d0103470 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009930 tx=0x7f68b802a420 comp rx=0 tx=0).stop 2024-05-17T04:51:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 -- 172.21.15.50:0/4267186758 shutdown_connections 2024-05-17T04:51:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 --2- 172.21.15.50:0/4267186758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103b00 0x7f68d010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 --2- 172.21.15.50:0/4267186758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103000 0x7f68d0103470 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 --2- 172.21.15.50:0/4267186758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 -- 172.21.15.50:0/4267186758 >> 172.21.15.50:0/4267186758 conn(0x7f68d00fd010 msgr2=0x7f68d00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 -- 172.21.15.50:0/4267186758 shutdown_connections 2024-05-17T04:51:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.865+0000 7f68d4e66640 1 -- 172.21.15.50:0/4267186758 wait complete. 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 Processor -- start 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 -- start start 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103000 0x7f68d019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 0x7f68d01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010da70 con 0x7f68d0103b00 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010d8f0 con 0x7f68d0102620 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.866+0000 7f68d4e66640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68d010dbf0 con 0x7f68d0103000 2024-05-17T04:51:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45142/0 (socket says 172.21.15.50:45142) 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 -- 172.21.15.50:0/166871112 learned_addr learned my addr 172.21.15.50:0/166871112 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ced9d640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 0x7f68d01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68c7fff640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103000 0x7f68d019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 -- 172.21.15.50:0/166871112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103000 msgr2=0x7f68d019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103000 0x7f68d019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 -- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 msgr2=0x7f68d01a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 0x7f68d01a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 -- 172.21.15.50:0/166871112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d01a1310 con 0x7f68d0102620 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ced9d640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 0x7f68d01a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.867+0000 7f68ce59c640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68c000eb90 tx=0x7f68c000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.868+0000 7f68c77fe640 1 -- 172.21.15.50:0/166871112 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0010630 con 0x7f68d0102620 2024-05-17T04:51:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.868+0000 7f68c77fe640 1 -- 172.21.15.50:0/166871112 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68c000ee90 con 0x7f68d0102620 2024-05-17T04:51:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.868+0000 7f68c77fe640 1 -- 172.21.15.50:0/166871112 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0018a50 con 0x7f68d0102620 2024-05-17T04:51:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.868+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d01a1600 con 0x7f68d0102620 2024-05-17T04:51:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.868+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d0198550 con 0x7f68d0102620 2024-05-17T04:51:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.870+0000 7f68dbdf4740 1 Processor -- start 2024-05-17T04:51:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 -- start start 2024-05-17T04:51:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a23420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562b04985cc0 con 0x562b046701b0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562b04975860 con 0x562b046472f0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562b049798a0 con 0x562b0464f4d0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68c77fe640 1 -- 172.21.15.50:0/166871112 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f68c0018bf0 con 0x7f68d0102620 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68ced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68ced9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36014/0 (socket says 172.21.15.50:36014) 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68ced9d640 1 -- 172.21.15.50:0/3213656846 learned_addr learned my addr 172.21.15.50:0/3213656846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a23420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.871+0000 7f68ce59c640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 -- 172.21.15.50:0/3213656846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 msgr2=0x562b04a1c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 -- 172.21.15.50:0/3213656846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 msgr2=0x562b04a1bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 -- 172.21.15.50:0/3213656846 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0493dd50 con 0x562b0464f4d0 2024-05-17T04:51:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68ced9d640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a23420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009900 tx=0x7f68b8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123ee6d8f6501a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c4ff9640 1 -- 172.21.15.50:0/3213656846 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8035030 con 0x562b0464f4d0 2024-05-17T04:51:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c4ff9640 1 -- 172.21.15.50:0/3213656846 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68b8040070 con 0x562b0464f4d0 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c4ff9640 1 -- 172.21.15.50:0/3213656846 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8003850 con 0x562b0464f4d0 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68c77fe640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 0x7f68bc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.872+0000 7f68ced9d640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 0x7f68bc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68c77fe640 1 -- 172.21.15.50:0/166871112 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f68c00e2bd0 con 0x7f68d0102620 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3213656846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 msgr2=0x562b04a23420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68ced9d640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 0x7f68bc0b4cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x562b04a1d170 tx=0x7f68c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a23420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009900 tx=0x7f68b8005fd0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3213656846 shutdown_connections 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a23420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3213656846 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.877 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.873+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3213656846 >> 172.21.15.50:0/3213656846 conn(0x562b04664300 msgr2=0x562b04a179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3213656846 shutdown_connections 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3213656846 wait complete. 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- start start 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 Processor -- start 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 0x562b04a25d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562b048b1ab0 con 0x562b046472f0 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562b049324d0 con 0x562b0464f4d0 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562b04a23e40 con 0x562b046701b0 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.874+0000 7f68dbdf4740 10 client.0 _ll_register_callbacks cb 0x562b04970950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.878 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ced9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36030/0 (socket says 172.21.15.50:36030) 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 -- 172.21.15.50:0/3657284899 learned_addr learned my addr 172.21.15.50:0/3657284899 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 0x562b04a25d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 -- 172.21.15.50:0/3657284899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 msgr2=0x562b04a2ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 -- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 msgr2=0x562b04a25d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 0x562b04a25d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 -- 172.21.15.50:0/3657284899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b04933340 con 0x562b046472f0 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 0x562b04a25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68ce59c640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a9bb0 tx=0x7f68c00a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce5ab7b012ba3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:51:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00ae1b0 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b8040070 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.875+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f68c00a64a0 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f68c00b2c60 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 10 client.16458 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562b046db790 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68b8040070 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f68c00c4120 con 0x562b046472f0 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 10 client.16458 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 20 client.16458 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b0462ed80 0x562b04a33080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 4 client.16458 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.876+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b04987c10 con 0x562b0462ed80 2024-05-17T04:51:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.877+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b0462ed80 0x562b04a33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.877+0000 7f68b5ffb640 10 client.16458 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.877+0000 7f68c7fff640 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b0462ed80 0x562b04a33080 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dafac1179d511a9 server_cookie=754a0fafcd71596e in_seq=0 out_seq=0 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.878+0000 7f68b5ffb640 1 -- 172.21.15.50:0/3657284899 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b04987c10 con 0x562b0462ed80 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.878+0000 7f68b5ffb640 10 client.16458 handle_command_reply: tid=0 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.878+0000 7f68dbdf4740 1 client.16458 shutdown 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 msgr2=0x562b04a25810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a9bb0 tx=0x7f68c00a98c0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 shutdown_connections 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b0464f4d0 0x562b04a25d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b0462ed80 0x562b04a33080 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/3657284899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046472f0 0x562b04a25810 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 >> 172.21.15.50:0/3657284899 conn(0x562b04664300 msgr2=0x562b04a0e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68b5ffb640 0 client.16458 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 shutdown_connections 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 1 -- 172.21.15.50:0/3657284899 wait complete. 2024-05-17T04:51:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.879+0000 7f68dbdf4740 20 client.16458 trim_cache size 0 max 16384 2024-05-17T04:51:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.905+0000 7f68dbdf4740 1 Processor -- start 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.905+0000 7f68dbdf4740 1 -- start start 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.905+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.905+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b0464f4d0 0x562b04a1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b0464f4d0 0x562b04a1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ac660 con 0x562b0464f4d0 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a9f30 con 0x562b046472f0 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45168/0 (socket says 172.21.15.50:45168) 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 -- 172.21.15.50:0/448898643 learned_addr learned my addr 172.21.15.50:0/448898643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a7d80 con 0x562b046701b0 2024-05-17T04:51:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ced9d640 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 -- 172.21.15.50:0/448898643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 msgr2=0x562b04a2cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 -- 172.21.15.50:0/448898643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b0464f4d0 msgr2=0x562b04a1b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b0464f4d0 0x562b04a1b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.906+0000 7f68ce59c640 1 -- 172.21.15.50:0/448898643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b04754620 con 0x562b046472f0 2024-05-17T04:51:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.907+0000 7f68ce59c640 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a6450 tx=0x7f68c0011fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc959a2a512b253 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.907+0000 7f68b7fff640 1 -- 172.21.15.50:0/448898643 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00af5a0 con 0x562b046472f0 2024-05-17T04:51:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.907+0000 7f68b7fff640 1 -- 172.21.15.50:0/448898643 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68c00c4120 con 0x562b046472f0 2024-05-17T04:51:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.907+0000 7f68b7fff640 1 -- 172.21.15.50:0/448898643 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00a8dc0 con 0x562b046472f0 2024-05-17T04:51:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 msgr2=0x562b04a1adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68c00a6450 tx=0x7f68c0011fe0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 shutdown_connections 2024-05-17T04:51:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b046701b0 0x562b04a2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b0464f4d0 0x562b04a1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/448898643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a1adf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 >> 172.21.15.50:0/448898643 conn(0x562b04664300 msgr2=0x562b04a1a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 shutdown_connections 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.908+0000 7f68dbdf4740 1 -- 172.21.15.50:0/448898643 wait complete. 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 -- start start 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 Processor -- start 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a29990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a83f0 con 0x562b046701b0 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a9860 con 0x562b046472f0 2024-05-17T04:51:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01a8ac0 con 0x562b0464f4d0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68dbdf4740 10 client.0 _ll_register_callbacks cb 0x7f68c00a9c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68ce59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:45176/0 (socket says 172.21.15.50:45176) 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68ce59c640 1 -- 172.21.15.50:0/2145124836 learned_addr learned my addr 172.21.15.50:0/2145124836 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.909+0000 7f68ced9d640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a29990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 -- 172.21.15.50:0/2145124836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 msgr2=0x562b04a29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 -- 172.21.15.50:0/2145124836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 msgr2=0x562b04a29990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a29990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 -- 172.21.15.50:0/2145124836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b04a2cf70 con 0x562b046472f0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68ce59c640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68c00c4620 tx=0x7f68c00af7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d267c479eb8ca194 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:51:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00c42d0 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f68c00a5a30 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c00c4120 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f68c00ae1b0 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68dbdf4740 10 client.26216 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.910+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562b04a9a270 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68c00c4120 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f68c0004c40 con 0x562b046472f0 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 10 client.26216 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 20 client.26216 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b046041c0 0x562b04a9cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 4 client.26216 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:51:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b049813f0 con 0x562b046041c0 2024-05-17T04:51:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.911+0000 7f68c7fff640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b046041c0 0x562b04a9cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.912+0000 7f68b5ffb640 10 client.26216 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.912+0000 7f68c7fff640 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b046041c0 0x562b04a9cac0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b567581ed1fa1d3a server_cookie=2f8c9658af937327 in_seq=0 out_seq=0 2024-05-17T04:51:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68b5ffb640 1 -- 172.21.15.50:0/2145124836 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b049813f0 con 0x562b046041c0 2024-05-17T04:51:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68b5ffb640 10 client.26216 handle_command_reply: tid=0 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 client.26216 shutdown 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 msgr2=0x562b04a2c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f68c00c4620 tx=0x7f68c00af7b0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 shutdown_connections 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x562b046041c0 0x562b04a9cac0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562b046701b0 0x562b04a29990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68b5ffb640 0 client.26216 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562b0464f4d0 0x562b04a29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 --2- 172.21.15.50:0/2145124836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562b046472f0 0x562b04a2c8d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 >> 172.21.15.50:0/2145124836 conn(0x562b04664300 msgr2=0x562b04a1b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 shutdown_connections 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.931+0000 7f68dbdf4740 1 -- 172.21.15.50:0/2145124836 wait complete. 2024-05-17T04:51:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.932+0000 7f68dbdf4740 20 client.26216 trim_cache size 0 max 16384 2024-05-17T04:51:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:51:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:51:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 msgr2=0x7f68bc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 0x7f68bc0b4cf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x562b04a1d170 tx=0x7f68c8005f90 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 msgr2=0x7f68d0101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68c000eb90 tx=0x7f68c000c6a0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 shutdown_connections 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f68bc0b27e0 0x7f68bc0b4cf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68d0103b00 0x7f68d01a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68d0103000 0x7f68d019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 --2- 172.21.15.50:0/166871112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d0102620 0x7f68d0101ce0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 >> 172.21.15.50:0/166871112 conn(0x7f68d00fd010 msgr2=0x7f68d0106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 shutdown_connections 2024-05-17T04:51:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:08.933+0000 7f68d4e66640 1 -- 172.21.15.50:0/166871112 wait complete. 2024-05-17T04:51:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:10 smithi105 ceph-mon[34795]: pgmap v884: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:51:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:10 smithi110 ceph-mon[35497]: pgmap v884: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:51:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:10 smithi050 ceph-mon[31739]: pgmap v884: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-05-17T04:51:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:12 smithi105 ceph-mon[34795]: pgmap v885: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-17T04:51:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:12 smithi110 ceph-mon[35497]: pgmap v885: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-17T04:51:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:12 smithi050 ceph-mon[31739]: pgmap v885: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-17T04:51:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 735.0 s, 115.8 tps, lat 875.790 ms stddev 212.679 2024-05-17T04:51:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:14 smithi105 ceph-mon[34795]: pgmap v886: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-17T04:51:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:14 smithi110 ceph-mon[35497]: pgmap v886: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-17T04:51:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:14 smithi050 ceph-mon[31739]: pgmap v886: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 1.9 MiB/s wr, 320 op/s 2024-05-17T04:51:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:16 smithi105 ceph-mon[34795]: pgmap v887: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-05-17T04:51:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:16 smithi110 ceph-mon[35497]: pgmap v887: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-05-17T04:51:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:16 smithi050 ceph-mon[31739]: pgmap v887: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.1 MiB/s wr, 349 op/s 2024-05-17T04:51:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 740.0 s, 102.8 tps, lat 973.724 ms stddev 220.452 2024-05-17T04:51:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:18 smithi105 ceph-mon[34795]: pgmap v888: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 2.0 MiB/s wr, 319 op/s 2024-05-17T04:51:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:18 smithi110 ceph-mon[35497]: pgmap v888: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 2.0 MiB/s wr, 319 op/s 2024-05-17T04:51:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:18 smithi050 ceph-mon[31739]: pgmap v888: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 2.0 MiB/s wr, 319 op/s 2024-05-17T04:51:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:20 smithi105 ceph-mon[34795]: pgmap v889: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:51:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:20 smithi110 ceph-mon[35497]: pgmap v889: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:51:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:20 smithi050 ceph-mon[31739]: pgmap v889: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 2.2 MiB/s wr, 346 op/s 2024-05-17T04:51:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 -- 172.21.15.50:0/877800713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 msgr2=0x7f1cc810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 --2- 172.21.15.50:0/877800713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc810aa40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009900 tx=0x7f1cbc02a410 comp rx=0 tx=0).stop 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 -- 172.21.15.50:0/877800713 shutdown_connections 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 --2- 172.21.15.50:0/877800713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 --2- 172.21.15.50:0/877800713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc810aa40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 --2- 172.21.15.50:0/877800713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 -- 172.21.15.50:0/877800713 >> 172.21.15.50:0/877800713 conn(0x7f1cc80fd880 msgr2=0x7f1cc80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 -- 172.21.15.50:0/877800713 shutdown_connections 2024-05-17T04:51:22.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.081+0000 7f1ccde64640 1 -- 172.21.15.50:0/877800713 wait complete. 2024-05-17T04:51:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.082+0000 7f1ccde64640 1 Processor -- start 2024-05-17T04:51:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 -- start start 2024-05-17T04:51:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc81a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8110d50 con 0x7f1cc8108620 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8110bd0 con 0x7f1cc806b230 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1ccde64640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8110ed0 con 0x7f1cc810b060 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56386/0 (socket says 172.21.15.50:56386) 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52770/0 (socket says 172.21.15.50:52770) 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc77fe640 1 -- 172.21.15.50:0/3298041060 learned_addr learned my addr 172.21.15.50:0/3298041060 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.083+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc81a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 -- 172.21.15.50:0/3298041060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 msgr2=0x7f1cc819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 -- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 msgr2=0x7f1cc819bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.085 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 -- 172.21.15.50:0/3298041060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc81a3560 con 0x7f1cc810b060 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc81a2ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fd0 tx=0x7f1cc000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc4ff9640 1 -- 172.21.15.50:0/3298041060 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000ce20 con 0x7f1cc810b060 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc4ff9640 1 -- 172.21.15.50:0/3298041060 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cc0005020 con 0x7f1cc810b060 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1cc4ff9640 1 -- 172.21.15.50:0/3298041060 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0010690 con 0x7f1cc810b060 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc81a3850 con 0x7f1cc810b060 2024-05-17T04:51:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.084+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81a3d50 con 0x7f1cc810b060 2024-05-17T04:51:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.087+0000 7f1cd4e03740 1 Processor -- start 2024-05-17T04:51:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.087+0000 7f1cc4ff9640 1 -- 172.21.15.50:0/3298041060 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1cc0010830 con 0x7f1cc810b060 2024-05-17T04:51:22.087 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 -- start start 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f29d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f29dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556f5f20fcc0 con 0x556f5eed94d0 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556f5f1ff860 con 0x556f5eed12f0 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f5f2038a0 con 0x556f5eefa1b0 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52776/0 (socket says 172.21.15.50:52776) 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/2878295539 learned_addr learned my addr 172.21.15.50:0/2878295539 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:22.089 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f29d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f29dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc4ff9640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 0x7f1cb00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/2878295539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 msgr2=0x556f5f29d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f29d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/2878295539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 msgr2=0x556f5f29dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f29dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.088+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/2878295539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5f1c7d50 con 0x556f5eed94d0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 0x7f1cb00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cc4ff9640 1 -- 172.21.15.50:0/3298041060 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1cc00e2bc0 con 0x7f1cc810b060 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031460 tx=0x7f1cbc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691ca6cd8ba513f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cadffb640 1 -- 172.21.15.50:0/2878295539 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc035040 con 0x556f5eed94d0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cadffb640 1 -- 172.21.15.50:0/2878295539 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cbc0044c0 con 0x556f5eed94d0 2024-05-17T04:51:22.090 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.089+0000 7f1cadffb640 1 -- 172.21.15.50:0/2878295539 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc003c40 con 0x556f5eed94d0 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 -- 172.21.15.50:0/2878295539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 msgr2=0x556f5f2a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc031460 tx=0x7f1cbc005fd0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cc7fff640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 0x7f1cb00b4cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0000f80 tx=0x7f1cc009e720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 -- 172.21.15.50:0/2878295539 shutdown_connections 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a4c10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f29dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/2878295539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f29d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 -- 172.21.15.50:0/2878295539 >> 172.21.15.50:0/2878295539 conn(0x556f5eeee300 msgr2=0x556f5f2991a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 -- 172.21.15.50:0/2878295539 shutdown_connections 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.090+0000 7f1cd4e03740 1 -- 172.21.15.50:0/2878295539 wait complete. 2024-05-17T04:51:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 -- start start 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 Processor -- start 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556f5f13bab0 con 0x556f5eed94d0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556f5f1bc4d0 con 0x556f5eed12f0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f5f2a5630 con 0x556f5eefa1b0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:56410/0 (socket says 172.21.15.50:56410) 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/1810304319 learned_addr learned my addr 172.21.15.50:0/1810304319 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.091+0000 7f1cd4e03740 10 client.0 _ll_register_callbacks cb 0x556f5f1fa950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc77fe640 1 -- 172.21.15.50:0/1810304319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 msgr2=0x556f5f2ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc77fe640 1 -- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 msgr2=0x556f5f2a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc77fe640 1 -- 172.21.15.50:0/1810304319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5f1bd340 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.092+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a7570 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556f5f29eb70 tx=0x7f1cb800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bef1d722f30db60 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80120d0 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f5f211c10 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1cb8004020 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1cb8025050 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1cd4e03740 10 client.16476 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f5ef65790 con 0x556f5eed94d0 2024-05-17T04:51:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.093+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556f5f211c10 con 0x556f5eed94d0 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb801c470 con 0x556f5eed94d0 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 10 client.16476 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 20 client.16476 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5eeb8d80 0x556f5f2b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 4 client.16476 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f5f20b3f0 con 0x556f5eeb8d80 2024-05-17T04:51:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5eeb8d80 0x556f5f2b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1ca6ffd640 10 client.16476 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.094+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5eeb8d80 0x556f5f2b4900 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f6cbb26f9650f3 server_cookie=f117bfcdd807a4fc in_seq=0 out_seq=0 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/1810304319 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f5f20b3f0 con 0x556f5eeb8d80 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1ca6ffd640 10 client.16476 handle_command_reply: tid=0 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 client.16476 shutdown 2024-05-17T04:51:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 msgr2=0x556f5f2a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a7570 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556f5f29eb70 tx=0x7f1cb800f790 comp rx=0 tx=0).stop 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 shutdown_connections 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.095+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5eeb8d80 0x556f5f2b4900 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed94d0 0x556f5f2a7570 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1810304319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eed12f0 0x556f5f2a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 >> 172.21.15.50:0/1810304319 conn(0x556f5eeee300 msgr2=0x556f5f29c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1ca6ffd640 0 client.16476 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 shutdown_connections 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1810304319 wait complete. 2024-05-17T04:51:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.096+0000 7f1cd4e03740 20 client.16476 trim_cache size 0 max 16384 2024-05-17T04:51:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.118+0000 7f1cd4e03740 1 Processor -- start 2024-05-17T04:51:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.118+0000 7f1cd4e03740 1 -- start start 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81b0b10 con 0x556f5eed12f0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ae3e0 con 0x556f5eefa1b0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ac230 con 0x556f5ee8e1c0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52790/0 (socket says 172.21.15.50:52790) 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/1970094352 learned_addr learned my addr 172.21.15.50:0/1970094352 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/1970094352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5ee8e1c0 msgr2=0x556f5f2ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/1970094352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 msgr2=0x556f5f2ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/1970094352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5efde620 con 0x556f5eed12f0 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.119+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0057c0 tx=0x7f1cbc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c306461695fa35 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.120+0000 7f1cad7fa640 1 -- 172.21.15.50:0/1970094352 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc035040 con 0x556f5eed12f0 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.120+0000 7f1cad7fa640 1 -- 172.21.15.50:0/1970094352 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cbc004150 con 0x556f5eed12f0 2024-05-17T04:51:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.120+0000 7f1cad7fa640 1 -- 172.21.15.50:0/1970094352 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0577c0 con 0x556f5eed12f0 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1970094352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 msgr2=0x556f5f2a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0057c0 tx=0x7f1cbc005d50 comp rx=0 tx=0).stop 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1970094352 shutdown_connections 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/1970094352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5eed12f0 0x556f5f2a0bb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1970094352 >> 172.21.15.50:0/1970094352 conn(0x556f5eeee300 msgr2=0x556f5f29c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1970094352 shutdown_connections 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- 172.21.15.50:0/1970094352 wait complete. 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.121+0000 7f1cd4e03740 1 -- start start 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cd4e03740 1 Processor -- start 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cd4e03740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cc6ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ac110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:52792/0 (socket says 172.21.15.50:52792) 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.122+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48372/0 (socket says 172.21.15.50:48372) 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81ac8a0 con 0x556f5ee8e1c0 2024-05-17T04:51:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81add10 con 0x556f5eefa1b0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cd4e03740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc81acf70 con 0x556f5eed12f0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc6ffd640 1 -- 172.21.15.50:0/614001364 learned_addr learned my addr 172.21.15.50:0/614001364 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cd4e03740 10 client.0 _ll_register_callbacks cb 0x7f1cb8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 -- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 msgr2=0x556f5f2ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 -- 172.21.15.50:0/614001364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 msgr2=0x556f5f2ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 -- 172.21.15.50:0/614001364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5f2adb00 con 0x556f5eed12f0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cc77fe640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x556f5f2a8430 tx=0x7f1cb801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e4c37fec2b4ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:51:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb802d020 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1cb8018070 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.123+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc004150 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1cb8000e60 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 10 client.26209 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f5f31ccb0 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cbc004150 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb8005040 con 0x556f5eed12f0 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 10 client.26209 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 20 client.26209 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5f31d540 0x556f5f31f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 4 client.26209 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.124+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f5f185750 con 0x556f5f31d540 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.125+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5f31d540 0x556f5f31f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.125+0000 7f1ca6ffd640 10 client.26209 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.125+0000 7f1cc6ffd640 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5f31d540 0x556f5f31f960 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41fc3c2525eea45 server_cookie=385ff9aae881698e in_seq=0 out_seq=0 2024-05-17T04:51:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.151+0000 7f1ca6ffd640 1 -- 172.21.15.50:0/614001364 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f5f185750 con 0x556f5f31d540 2024-05-17T04:51:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.151+0000 7f1ca6ffd640 10 client.26209 handle_command_reply: tid=0 2024-05-17T04:51:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.151+0000 7f1cd4e03740 1 client.26209 shutdown 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 msgr2=0x556f5f2ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x556f5f2a8430 tx=0x7f1cb801c470 comp rx=0 tx=0).stop 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 shutdown_connections 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f5eefa1b0 0x556f5f2ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f5eed12f0 0x556f5f2ac650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x556f5f31d540 0x556f5f31f960 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1ca6ffd640 0 client.26209 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 --2- 172.21.15.50:0/614001364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f5ee8e1c0 0x556f5f2ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 >> 172.21.15.50:0/614001364 conn(0x556f5eeee300 msgr2=0x556f5f29c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 shutdown_connections 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.152+0000 7f1cd4e03740 1 -- 172.21.15.50:0/614001364 wait complete. 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.153+0000 7f1cd4e03740 20 client.26209 trim_cache size 0 max 16384 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:51:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.153+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 msgr2=0x7f1cb00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.153+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 0x7f1cb00b4cf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0000f80 tx=0x7f1cc009e720 comp rx=0 tx=0).stop 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.153+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 msgr2=0x7f1cc81a2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc81a2ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fd0 tx=0x7f1cc000ef10 comp rx=0 tx=0).stop 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 shutdown_connections 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cc810b060 0x7f1cc81a2ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f1cb00b27e0 0x7f1cb00b4cf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cc8108620 0x7f1cc819bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 --2- 172.21.15.50:0/3298041060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc806b230 0x7f1cc819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 >> 172.21.15.50:0/3298041060 conn(0x7f1cc80fd880 msgr2=0x7f1cc81092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 shutdown_connections 2024-05-17T04:51:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:22.154+0000 7f1ccde64640 1 -- 172.21.15.50:0/3298041060 wait complete. 2024-05-17T04:51:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:22 smithi105 ceph-mon[34795]: pgmap v890: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-05-17T04:51:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:22 smithi110 ceph-mon[35497]: pgmap v890: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-05-17T04:51:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:22 smithi050 ceph-mon[31739]: pgmap v890: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.2 MiB/s wr, 337 op/s 2024-05-17T04:51:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 745.0 s, 104.0 tps, lat 933.042 ms stddev 238.864 2024-05-17T04:51:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:24 smithi105 ceph-mon[34795]: pgmap v891: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:51:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:24 smithi110 ceph-mon[35497]: pgmap v891: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:51:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:24 smithi050 ceph-mon[31739]: pgmap v891: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:51:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:26 smithi105 ceph-mon[34795]: pgmap v892: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.5 MiB/s wr, 366 op/s 2024-05-17T04:51:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:26 smithi110 ceph-mon[35497]: pgmap v892: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.5 MiB/s wr, 366 op/s 2024-05-17T04:51:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:26 smithi050 ceph-mon[31739]: pgmap v892: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.5 MiB/s wr, 366 op/s 2024-05-17T04:51:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 750.0 s, 111.8 tps, lat 914.201 ms stddev 352.287 2024-05-17T04:51:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:28 smithi105 ceph-mon[34795]: pgmap v893: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:28 smithi110 ceph-mon[35497]: pgmap v893: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:28 smithi050 ceph-mon[31739]: pgmap v893: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:30 smithi105 ceph-mon[34795]: pgmap v894: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.8 MiB/s wr, 360 op/s 2024-05-17T04:51:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:30 smithi110 ceph-mon[35497]: pgmap v894: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.8 MiB/s wr, 360 op/s 2024-05-17T04:51:30.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:30 smithi050 ceph-mon[31739]: pgmap v894: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.8 MiB/s wr, 360 op/s 2024-05-17T04:51:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:32 smithi105 ceph-mon[34795]: pgmap v895: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-05-17T04:51:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:32 smithi110 ceph-mon[35497]: pgmap v895: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-05-17T04:51:32.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:32 smithi050 ceph-mon[31739]: pgmap v895: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 331 op/s 2024-05-17T04:51:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 755.0 s, 105.0 tps, lat 949.012 ms stddev 222.840 2024-05-17T04:51:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:34 smithi105 ceph-mon[34795]: pgmap v896: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:51:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:34 smithi110 ceph-mon[35497]: pgmap v896: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:51:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:34 smithi050 ceph-mon[31739]: pgmap v896: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 -- 172.21.15.50:0/1491854568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 msgr2=0x7f2158105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 --2- 172.21.15.50:0/1491854568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158105d20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2154009900 tx=0x7f215402a410 comp rx=0 tx=0).stop 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 -- 172.21.15.50:0/1491854568 shutdown_connections 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 --2- 172.21.15.50:0/1491854568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 0x7f215810b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 --2- 172.21.15.50:0/1491854568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 --2- 172.21.15.50:0/1491854568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158105d20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 -- 172.21.15.50:0/1491854568 >> 172.21.15.50:0/1491854568 conn(0x7f21580fd880 msgr2=0x7f21580ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 -- 172.21.15.50:0/1491854568 shutdown_connections 2024-05-17T04:51:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.287+0000 7f215f817640 1 -- 172.21.15.50:0/1491854568 wait complete. 2024-05-17T04:51:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 Processor -- start 2024-05-17T04:51:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 -- start start 2024-05-17T04:51:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 0x7f21581a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f215810ebb0 con 0x7f21581062f0 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f215810ea30 con 0x7f2158108e30 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215f817640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f215810ed30 con 0x7f215806b1f0 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215d5b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40058/0 (socket says 172.21.15.50:40058) 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 0x7f21581a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 -- 172.21.15.50:0/4173199817 learned_addr learned my addr 172.21.15.50:0/4173199817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 -- 172.21.15.50:0/4173199817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 msgr2=0x7f2158199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 -- 172.21.15.50:0/4173199817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 msgr2=0x7f21581a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 0x7f21581a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.288+0000 7f215cdb2640 1 -- 172.21.15.50:0/4173199817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21581a13a0 con 0x7f21581062f0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.289+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9c0 tx=0x7f214800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.289+0000 7f214e7fc640 1 -- 172.21.15.50:0/4173199817 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800cd80 con 0x7f21581062f0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.289+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2158101b60 con 0x7f21581062f0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.289+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2158102100 con 0x7f21581062f0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.289+0000 7f214e7fc640 1 -- 172.21.15.50:0/4173199817 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21480054b0 con 0x7f21581062f0 2024-05-17T04:51:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.290+0000 7f214e7fc640 1 -- 172.21.15.50:0/4173199817 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148004da0 con 0x7f21581062f0 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 Processor -- start 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 -- start start 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d1dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d24c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ef7c8fcc0 con 0x555ef79594d0 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555ef7c7f860 con 0x555ef797a1b0 2024-05-17T04:51:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.292+0000 7f21667aa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555ef7c838a0 con 0x555ef79512f0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49790/0 (socket says 172.21.15.50:49790) 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 -- 172.21.15.50:0/2206410404 learned_addr learned my addr 172.21.15.50:0/2206410404 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215d5b3640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d1dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f214e7fc640 1 -- 172.21.15.50:0/4173199817 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2148005020 con 0x7f21581062f0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 -- 172.21.15.50:0/2206410404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 msgr2=0x555ef7d1dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d1dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 -- 172.21.15.50:0/2206410404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 msgr2=0x555ef7d24c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215cdb2640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d24c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d24c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215ddb4640 1 -- 172.21.15.50:0/2206410404 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef7c47d50 con 0x555ef797a1b0 2024-05-17T04:51:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.293+0000 7f215cdb2640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d24c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:51:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f215ddb4640 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2150009fd0 tx=0x7f21500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca6b66f0f8bc74c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f21437fe640 1 -- 172.21.15.50:0/2206410404 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150004d20 con 0x555ef797a1b0 2024-05-17T04:51:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f21437fe640 1 -- 172.21.15.50:0/2206410404 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2150004050 con 0x555ef797a1b0 2024-05-17T04:51:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f214e7fc640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 0x7f21440b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f215d5b3640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 0x7f21440b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f214e7fc640 1 -- 172.21.15.50:0/4173199817 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f21480e28d0 con 0x7f21581062f0 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.294+0000 7f21437fe640 1 -- 172.21.15.50:0/2206410404 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150005260 con 0x555ef797a1b0 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 -- 172.21.15.50:0/2206410404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 msgr2=0x555ef7d1d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f215d5b3640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 0x7f21440b4d70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21540057c0 tx=0x7f2154031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2150009fd0 tx=0x7f21500118d0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 -- 172.21.15.50:0/2206410404 shutdown_connections 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d24c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d1dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 --2- 172.21.15.50:0/2206410404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1d770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 -- 172.21.15.50:0/2206410404 >> 172.21.15.50:0/2206410404 conn(0x555ef796e300 msgr2=0x555ef7d191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 -- 172.21.15.50:0/2206410404 shutdown_connections 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.295+0000 7f21667aa740 1 -- 172.21.15.50:0/2206410404 wait complete. 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 -- start start 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 Processor -- start 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef797a1b0 0x555ef7d2e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ef7bbbab0 con 0x555ef79594d0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555ef7c3c4d0 con 0x555ef79512f0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555ef7d25730 con 0x555ef797a1b0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f21667aa740 10 client.0 _ll_register_callbacks cb 0x555ef7c7a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f215d5b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f215d5b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49796/0 (socket says 172.21.15.50:49796) 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef797a1b0 0x555ef7d2e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f215d5b3640 1 -- 172.21.15.50:0/2547929526 learned_addr learned my addr 172.21.15.50:0/2547929526 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.296+0000 7f215cdb2640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 -- 172.21.15.50:0/2547929526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef797a1b0 msgr2=0x555ef7d2e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef797a1b0 0x555ef7d2e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 -- 172.21.15.50:0/2547929526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 msgr2=0x555ef7d27600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d27600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 -- 172.21.15.50:0/2547929526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef7c3d340 con 0x555ef79512f0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f215d5b3640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2154006010 tx=0x7f215402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124527cd2b5bbca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f2140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154056070 con 0x555ef79512f0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2150004050 con 0x555ef79512f0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.297+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2154052850 con 0x555ef79512f0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2154062050 con 0x555ef79512f0 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 10 client.26243 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555ef79e5790 con 0x555ef79512f0 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2150004050 con 0x555ef79512f0 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2154031b30 con 0x555ef79512f0 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 10 client.26243 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 20 client.26243 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef7938d80 0x555ef7d34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.298+0000 7f21667aa740 4 client.26243 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.299+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef7c91c10 con 0x555ef7938d80 2024-05-17T04:51:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.299+0000 7f215cdb2640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef7938d80 0x555ef7d34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.300+0000 7f215cdb2640 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef7938d80 0x555ef7d34d00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d6e05824ed3cdc server_cookie=69a713618938334d in_seq=0 out_seq=0 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.300+0000 7f2140ff9640 10 client.26243 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.300+0000 7f2140ff9640 1 -- 172.21.15.50:0/2547929526 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ef7c91c10 con 0x555ef7938d80 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.300+0000 7f2140ff9640 10 client.26243 handle_command_reply: tid=0 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.300+0000 7f21667aa740 1 client.26243 shutdown 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 msgr2=0x555ef7d270c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2154006010 tx=0x7f215402a410 comp rx=0 tx=0).stop 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 shutdown_connections 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef797a1b0 0x555ef7d2e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef7938d80 0x555ef7d34d00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d27600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 --2- 172.21.15.50:0/2547929526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79512f0 0x555ef7d270c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 >> 172.21.15.50:0/2547929526 conn(0x555ef796e300 msgr2=0x555ef7d1c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f2140ff9640 0 client.26243 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 shutdown_connections 2024-05-17T04:51:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.301+0000 7f21667aa740 1 -- 172.21.15.50:0/2547929526 wait complete. 2024-05-17T04:51:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.302+0000 7f21667aa740 20 client.26243 trim_cache size 0 max 16384 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 Processor -- start 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 -- start start 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21581ac620 con 0x555ef79594d0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21581a9ef0 con 0x555ef797a1b0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f21667aa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21581a7d40 con 0x555ef79512f0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40082/0 (socket says 172.21.15.50:40082) 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49806/0 (socket says 172.21.15.50:49806) 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215cdb2640 1 -- 172.21.15.50:0/4058392678 learned_addr learned my addr 172.21.15.50:0/4058392678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.324+0000 7f215d5b3640 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 -- 172.21.15.50:0/4058392678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 msgr2=0x555ef7d192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 -- 172.21.15.50:0/4058392678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 msgr2=0x555ef7d1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 -- 172.21.15.50:0/4058392678 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef7a5e620 con 0x555ef79594d0 2024-05-17T04:51:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215ddb4640 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f215cdb2640 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f21480a9af0 tx=0x7f21480e4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46e13f15c08cc84 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f2142ffd640 1 -- 172.21.15.50:0/4058392678 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480a9670 con 0x555ef79594d0 2024-05-17T04:51:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f2142ffd640 1 -- 172.21.15.50:0/4058392678 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f214808f070 con 0x555ef79594d0 2024-05-17T04:51:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.325+0000 7f2142ffd640 1 -- 172.21.15.50:0/4058392678 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148005660 con 0x555ef79594d0 2024-05-17T04:51:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 -- 172.21.15.50:0/4058392678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 msgr2=0x555ef7d1bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f21480a9af0 tx=0x7f21480e4de0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 -- 172.21.15.50:0/4058392678 shutdown_connections 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef797a1b0 0x555ef7d1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef79594d0 0x555ef7d1bb50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 --2- 172.21.15.50:0/4058392678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 -- 172.21.15.50:0/4058392678 >> 172.21.15.50:0/4058392678 conn(0x555ef796e300 msgr2=0x555ef7d18bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 -- 172.21.15.50:0/4058392678 shutdown_connections 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.326+0000 7f21667aa740 1 -- 172.21.15.50:0/4058392678 wait complete. 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 -- start start 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 Processor -- start 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.327+0000 7f21667aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21581a83b0 con 0x555ef797a1b0 2024-05-17T04:51:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f21667aa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21581a9820 con 0x555ef79594d0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f21667aa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f21581a8a80 con 0x555ef79512f0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f21667aa740 10 client.0 _ll_register_callbacks cb 0x7f2154009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215ddb4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40094/0 (socket says 172.21.15.50:40094) 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215ddb4640 1 -- 172.21.15.50:0/3265295715 learned_addr learned my addr 172.21.15.50:0/3265295715 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49816/0 (socket says 172.21.15.50:49816) 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215d5b3640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 -- 172.21.15.50:0/3265295715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 msgr2=0x555ef7d39620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d39620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 -- 172.21.15.50:0/3265295715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 msgr2=0x555ef7d9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 -- 172.21.15.50:0/3265295715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef7d3a400 con 0x555ef79594d0 2024-05-17T04:51:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215ddb4640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f215cdb2640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21480a9af0 tx=0x7f2148000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dabd1bf47ffba309 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f2140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.328+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480c2020 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f21480a5df0 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214808f070 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f21480a9670 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 10 client.26249 fetch_fsmap learned FSMap version 41 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555ef7d9c590 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f214808f070 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f21480a54d0 con 0x555ef79594d0 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 10 client.26249 fetch_fsmap finished waiting for FSMap version 41 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 20 client.26249 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef790e1c0 0x555ef7d9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 4 client.26249 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:51:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ef7c8b3f0 con 0x555ef790e1c0 2024-05-17T04:51:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.329+0000 7f215d5b3640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef790e1c0 0x555ef7d9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.330+0000 7f2140ff9640 10 client.26249 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:35.334 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.330+0000 7f215d5b3640 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef790e1c0 0x555ef7d9ef80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767445393d0a30df server_cookie=7bf789d58117df9e in_seq=0 out_seq=0 2024-05-17T04:51:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f2140ff9640 1 -- 172.21.15.50:0/3265295715 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555ef7c8b3f0 con 0x555ef790e1c0 2024-05-17T04:51:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f2140ff9640 10 client.26249 handle_command_reply: tid=0 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 client.26249 shutdown 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 msgr2=0x555ef7d39b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21480a9af0 tx=0x7f2148000bc0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 shutdown_connections 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x555ef790e1c0 0x555ef7d9ef80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555ef797a1b0 0x555ef7d9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555ef79594d0 0x555ef7d39b60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 --2- 172.21.15.50:0/3265295715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555ef79512f0 0x555ef7d39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 >> 172.21.15.50:0/3265295715 conn(0x555ef796e300 msgr2=0x555ef7d1c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.356+0000 7f2140ff9640 0 client.26249 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 shutdown_connections 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f21667aa740 1 -- 172.21.15.50:0/3265295715 wait complete. 2024-05-17T04:51:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f21667aa740 20 client.26249 trim_cache size 0 max 16384 2024-05-17T04:51:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:51:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:51:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 msgr2=0x7f21440b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 0x7f21440b4d70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21540057c0 tx=0x7f2154031400 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 msgr2=0x7f2158199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9c0 tx=0x7f214800ee80 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 shutdown_connections 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2158108e30 0x7f21581a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f21440b2860 0x7f21440b4d70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21581062f0 0x7f2158199da0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 --2- 172.21.15.50:0/4173199817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f215806b1f0 0x7f2158199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.357+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 >> 172.21.15.50:0/4173199817 conn(0x7f21580fd880 msgr2=0x7f2158104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.358+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 shutdown_connections 2024-05-17T04:51:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:35.358+0000 7f215f817640 1 -- 172.21.15.50:0/4173199817 wait complete. 2024-05-17T04:51:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:36 smithi105 ceph-mon[34795]: pgmap v897: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-05-17T04:51:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:36 smithi110 ceph-mon[35497]: pgmap v897: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-05-17T04:51:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:36 smithi050 ceph-mon[31739]: pgmap v897: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-05-17T04:51:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 760.0 s, 99.4 tps, lat 997.191 ms stddev 237.208 2024-05-17T04:51:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:38 smithi105 ceph-mon[34795]: pgmap v898: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:38 smithi110 ceph-mon[35497]: pgmap v898: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:38 smithi050 ceph-mon[31739]: pgmap v898: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.5 MiB/s wr, 319 op/s 2024-05-17T04:51:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:40 smithi105 ceph-mon[34795]: pgmap v899: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:51:40.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:40 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:40 smithi110 ceph-mon[35497]: pgmap v899: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:51:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:40 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:40 smithi050 ceph-mon[31739]: pgmap v899: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:51:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:40 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:41.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:41 smithi105 ceph-mon[34795]: pgmap v900: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 319 op/s 2024-05-17T04:51:41.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:41 smithi110 ceph-mon[35497]: pgmap v900: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 319 op/s 2024-05-17T04:51:41.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:41 smithi050 ceph-mon[31739]: pgmap v900: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 319 op/s 2024-05-17T04:51:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 765.0 s, 94.4 tps, lat 1034.674 ms stddev 239.334 2024-05-17T04:51:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:44 smithi105 ceph-mon[34795]: pgmap v901: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.4 MiB/s wr, 333 op/s 2024-05-17T04:51:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:44 smithi110 ceph-mon[35497]: pgmap v901: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.4 MiB/s wr, 333 op/s 2024-05-17T04:51:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:44 smithi050 ceph-mon[31739]: pgmap v901: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.4 MiB/s wr, 333 op/s 2024-05-17T04:51:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:46 smithi105 ceph-mon[34795]: pgmap v902: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:51:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:46 smithi110 ceph-mon[35497]: pgmap v902: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:51:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:46 smithi050 ceph-mon[31739]: pgmap v902: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:51:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 770.0 s, 98.0 tps, lat 1032.133 ms stddev 262.419 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.474+0000 7fc8595a8640 1 -- 172.21.15.50:0/1421124079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 msgr2=0x7fc8541035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.474+0000 7fc8595a8640 1 --2- 172.21.15.50:0/1421124079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc8541035b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc84800b9f0 tx=0x7fc84802c0a0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 -- 172.21.15.50:0/1421124079 shutdown_connections 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 --2- 172.21.15.50:0/1421124079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854103af0 0x7fc85410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 --2- 172.21.15.50:0/1421124079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc8541035b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 --2- 172.21.15.50:0/1421124079 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854102080 0x7fc854102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 -- 172.21.15.50:0/1421124079 >> 172.21.15.50:0/1421124079 conn(0x7fc8540fd880 msgr2=0x7fc8540ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 -- 172.21.15.50:0/1421124079 shutdown_connections 2024-05-17T04:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.475+0000 7fc8595a8640 1 -- 172.21.15.50:0/1421124079 wait complete. 2024-05-17T04:51:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 Processor -- start 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 -- start start 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc854197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854103af0 0x7fc85419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:36104/0 (socket says 172.21.15.50:36104) 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc852ffd640 1 -- 172.21.15.50:0/2749329499 learned_addr learned my addr 172.21.15.50:0/2749329499 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410c960 con 0x7fc854103af0 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410c7e0 con 0x7fc854102080 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410cae0 con 0x7fc8541031a0 2024-05-17T04:51:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8527fc640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc854197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.476+0000 7fc8537fe640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854103af0 0x7fc85419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 -- 172.21.15.50:0/2749329499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 msgr2=0x7fc854197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc854197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 -- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854103af0 msgr2=0x7fc85419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854103af0 0x7fc85419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85419f150 con 0x7fc854102080 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc8527fc640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc854197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc852ffd640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e9f0 tx=0x7fc84400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc83bfff640 1 -- 172.21.15.50:0/2749329499 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84400ccc0 con 0x7fc854102080 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85419f440 con 0x7fc854102080 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc83bfff640 1 -- 172.21.15.50:0/2749329499 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8440108c0 con 0x7fc854102080 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc83bfff640 1 -- 172.21.15.50:0/2749329499 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440187c0 con 0x7fc854102080 2024-05-17T04:51:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.477+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85419f990 con 0x7fc854102080 2024-05-17T04:51:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.480+0000 7fc83bfff640 1 -- 172.21.15.50:0/2749329499 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc844018960 con 0x7fc854102080 2024-05-17T04:51:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 Processor -- start 2024-05-17T04:51:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 -- start start 2024-05-17T04:51:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc83bfff640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 0x7fc83c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa1dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa24c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5da98fcc0 con 0x55c5da6512f0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c5da97f860 con 0x55c5da6594d0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc860463740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5da9838a0 con 0x55c5da67a1b0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc8537fe640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 0x7fc83c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc8527fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa1dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.481+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52946/0 (socket says 172.21.15.50:52946) 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 -- 172.21.15.50:0/1071695116 learned_addr learned my addr 172.21.15.50:0/1071695116 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc83bfff640 1 -- 172.21.15.50:0/2749329499 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc8440e27f0 con 0x7fc854102080 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc8537fe640 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa24c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 -- 172.21.15.50:0/1071695116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 msgr2=0x55c5daa24c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa24c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 -- 172.21.15.50:0/1071695116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 msgr2=0x55c5daa1dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa1dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 -- 172.21.15.50:0/1071695116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5da947d50 con 0x55c5da67a1b0 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc8537fe640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 0x7fc83c0b4cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc852ffd640 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8440aa970 tx=0x7fc8440aa280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be94ae8d2c73d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc8397fa640 1 -- 172.21.15.50:0/1071695116 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440aa4f0 con 0x55c5da67a1b0 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc8397fa640 1 -- 172.21.15.50:0/1071695116 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8440aa9e0 con 0x55c5da67a1b0 2024-05-17T04:51:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.482+0000 7fc8397fa640 1 -- 172.21.15.50:0/1071695116 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440a6d60 con 0x55c5da67a1b0 2024-05-17T04:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 -- 172.21.15.50:0/1071695116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 msgr2=0x55c5daa1d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8440aa970 tx=0x7fc8440aa280 comp rx=0 tx=0).stop 2024-05-17T04:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 -- 172.21.15.50:0/1071695116 shutdown_connections 2024-05-17T04:51:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa24c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa1dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 --2- 172.21.15.50:0/1071695116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa1d770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.483+0000 7fc860463740 1 -- 172.21.15.50:0/1071695116 >> 172.21.15.50:0/1071695116 conn(0x55c5da66e300 msgr2=0x55c5daa191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 -- 172.21.15.50:0/1071695116 shutdown_connections 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 -- 172.21.15.50:0/1071695116 wait complete. 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 -- start start 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 Processor -- start 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa27550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.484+0000 7fc860463740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c5da8bbab0 con 0x55c5da6594d0 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc860463740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c5da93c4d0 con 0x55c5da67a1b0 2024-05-17T04:51:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc860463740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c5daa25690 con 0x55c5da6512f0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc860463740 10 client.0 _ll_register_callbacks cb 0x55c5da97a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc8527fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa27550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52950/0 (socket says 172.21.15.50:52950) 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 -- 172.21.15.50:0/4050995065 learned_addr learned my addr 172.21.15.50:0/4050995065 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc8537fe640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 -- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 msgr2=0x55c5daa2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 -- 172.21.15.50:0/4050995065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 msgr2=0x55c5daa27550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa27550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 -- 172.21.15.50:0/4050995065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5da93d340 con 0x55c5da6512f0 2024-05-17T04:51:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc852ffd640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8440dffb0 tx=0x7fc8440a6f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363d743a203e01b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440aa4f0 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.485+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc84408d430 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8440aa9e0 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc8440b28e0 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 10 client.26239 fetch_fsmap learned FSMap version 42 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c5da6e5790 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8440aa9e0 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7fc8440cc070 con 0x55c5da6512f0 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 10 client.26239 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 20 client.26239 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da638d80 0x55c5daa387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 4 client.26239 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.486+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5da991c10 con 0x55c5da638d80 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.487+0000 7fc8527fc640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da638d80 0x55c5daa387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.487+0000 7fc8527fc640 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da638d80 0x55c5daa387a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae4c47216aa2d7f server_cookie=4c8482c5ed3a7452 in_seq=0 out_seq=0 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.487+0000 7fc836ff5640 10 client.26239 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc836ff5640 1 -- 172.21.15.50:0/4050995065 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c5da991c10 con 0x55c5da638d80 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc836ff5640 10 client.26239 handle_command_reply: tid=0 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 client.26239 shutdown 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 msgr2=0x55c5daa27010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8440dffb0 tx=0x7fc8440a6f10 comp rx=0 tx=0).stop 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 shutdown_connections 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da638d80 0x55c5daa387a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa27550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc836ff5640 0 client.26239 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 --2- 172.21.15.50:0/4050995065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa27010 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 >> 172.21.15.50:0/4050995065 conn(0x55c5da66e300 msgr2=0x55c5daa181a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 shutdown_connections 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 1 -- 172.21.15.50:0/4050995065 wait complete. 2024-05-17T04:51:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.488+0000 7fc860463740 20 client.26239 trim_cache size 0 max 16384 2024-05-17T04:51:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 Processor -- start 2024-05-17T04:51:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 -- start start 2024-05-17T04:51:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa17ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc8400123f0 con 0x55c5da6594d0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc84000fcc0 con 0x55c5da67a1b0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc860463740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc84000db10 con 0x55c5da6512f0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc8527fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc8527fc640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41374/0 (socket says 172.21.15.50:41374) 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc8527fc640 1 -- 172.21.15.50:0/595697551 learned_addr learned my addr 172.21.15.50:0/595697551 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc852ffd640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa17ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.508+0000 7fc8537fe640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 -- 172.21.15.50:0/595697551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 msgr2=0x55c5daa17ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa17ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 -- 172.21.15.50:0/595697551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 msgr2=0x55c5daa21070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa21070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 -- 172.21.15.50:0/595697551 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5da75e620 con 0x55c5da6594d0 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8537fe640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc8527fc640 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc84802c5c0 tx=0x7fc848005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb972b2c94fe08e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc838ff9640 1 -- 172.21.15.50:0/595697551 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848039040 con 0x55c5da6594d0 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc838ff9640 1 -- 172.21.15.50:0/595697551 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8480054d0 con 0x55c5da6594d0 2024-05-17T04:51:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.509+0000 7fc838ff9640 1 -- 172.21.15.50:0/595697551 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848004c00 con 0x55c5da6594d0 2024-05-17T04:51:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 -- 172.21.15.50:0/595697551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 msgr2=0x55c5daa20b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc84802c5c0 tx=0x7fc848005c90 comp rx=0 tx=0).stop 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 -- 172.21.15.50:0/595697551 shutdown_connections 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da67a1b0 0x55c5daa21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6594d0 0x55c5daa20b30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 --2- 172.21.15.50:0/595697551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da6512f0 0x55c5daa17ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.510+0000 7fc860463740 1 -- 172.21.15.50:0/595697551 >> 172.21.15.50:0/595697551 conn(0x55c5da66e300 msgr2=0x55c5daa177e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- 172.21.15.50:0/595697551 shutdown_connections 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- 172.21.15.50:0/595697551 wait complete. 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- start start 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 Processor -- start 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa29420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa2c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc84000e180 con 0x55c5da6512f0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc84000f5f0 con 0x55c5da6594d0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc860463740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc84000e850 con 0x55c5da67a1b0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.511+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc860463740 10 client.0 _ll_register_callbacks cb 0x7fc844090f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa29420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41388/0 (socket says 172.21.15.50:41388) 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc852ffd640 1 -- 172.21.15.50:0/806751048 learned_addr learned my addr 172.21.15.50:0/806751048 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8537fe640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa2c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 -- 172.21.15.50:0/806751048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 msgr2=0x55c5daa2c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa2c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 -- 172.21.15.50:0/806751048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 msgr2=0x55c5daa28ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 -- 172.21.15.50:0/806751048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5daa2c950 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc852ffd640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc8527fc640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa29420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55c5daa14b90 tx=0x7fc848004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837a5c579ea1ca33 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848058030 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8480054d0 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.512+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc848009c10 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc848062050 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 10 client.26267 fetch_fsmap learned FSMap version 42 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c5daa2dbe0 con 0x55c5da6594d0 2024-05-17T04:51:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8480054d0 con 0x55c5da6594d0 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 42) ==== 8350+0+0 (secure 0 0 0) 0x7fc848049030 con 0x55c5da6594d0 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 10 client.26267 fetch_fsmap finished waiting for FSMap version 42 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 20 client.26267 populate_metadata read hostname 'smithi050' 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da60e1c0 0x55c5daaa2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 4 client.26267 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5da98b3f0 con 0x55c5da60e1c0 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.513+0000 7fc852ffd640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da60e1c0 0x55c5daaa2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.514+0000 7fc836ff5640 10 client.26267 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.514+0000 7fc852ffd640 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da60e1c0 0x55c5daaa2f60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54597342280abdf server_cookie=dafa6a5e70d48bec in_seq=0 out_seq=0 2024-05-17T04:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc836ff5640 1 -- 172.21.15.50:0/806751048 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c5da98b3f0 con 0x55c5da60e1c0 2024-05-17T04:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc836ff5640 10 client.26267 handle_command_reply: tid=0 2024-05-17T04:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 client.26267 shutdown 2024-05-17T04:51:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 msgr2=0x55c5daa29420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa29420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55c5daa14b90 tx=0x7fc848004700 comp rx=0 tx=0).stop 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 shutdown_connections 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c5da67a1b0 0x55c5daa2c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c5da6594d0 0x55c5daa29420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c5da60e1c0 0x55c5daaa2f60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 --2- 172.21.15.50:0/806751048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c5da6512f0 0x55c5daa28ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc836ff5640 0 client.26267 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 >> 172.21.15.50:0/806751048 conn(0x55c5da66e300 msgr2=0x55c5daa181a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 shutdown_connections 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.539+0000 7fc860463740 1 -- 172.21.15.50:0/806751048 wait complete. 2024-05-17T04:51:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.540+0000 7fc860463740 20 client.26267 trim_cache size 0 max 16384 2024-05-17T04:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:51:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.540+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 msgr2=0x7fc83c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.540+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 0x7fc83c0b4cf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c011bc0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.540+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 msgr2=0x7fc854197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.540+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e9f0 tx=0x7fc84400eeb0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 shutdown_connections 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc83c0b27e0 0x7fc83c0b4cf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc854103af0 0x7fc85419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8541031a0 0x7fc854197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 --2- 172.21.15.50:0/2749329499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc854102080 0x7fc854197610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 >> 172.21.15.50:0/2749329499 conn(0x7fc8540fd880 msgr2=0x7fc854106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 shutdown_connections 2024-05-17T04:51:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:51:48.541+0000 7fc8595a8640 1 -- 172.21.15.50:0/2749329499 wait complete. 2024-05-17T04:51:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:48 smithi105 ceph-mon[34795]: pgmap v903: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-17T04:51:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:48 smithi110 ceph-mon[35497]: pgmap v903: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-17T04:51:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:48 smithi050 ceph-mon[31739]: pgmap v903: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-05-17T04:51:49.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:49 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:51:49.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:49 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:51:49.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:49 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: pgmap v904: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.4 MiB/s wr, 335 op/s 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:51:50.477 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:51:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: pgmap v904: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.4 MiB/s wr, 335 op/s 2024-05-17T04:51:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:51:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:51:50.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: pgmap v904: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.4 MiB/s wr, 335 op/s 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:51:50.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:51:51.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:51 smithi105 ceph-mon[34795]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:51:51.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:51 smithi105 ceph-mon[34795]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:51:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:51 smithi110 ceph-mon[35497]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:51:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:51 smithi110 ceph-mon[35497]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:51:51.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:51 smithi050 ceph-mon[31739]: Adjusting osd_memory_target on smithi110 to 176.1M 2024-05-17T04:51:51.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:51 smithi050 ceph-mon[31739]: Unable to set osd_memory_target on smithi110 to 184745881: error parsing value: Value '184745881' is below minimum 939524096 2024-05-17T04:51:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:52 smithi105 ceph-mon[34795]: pgmap v905: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-05-17T04:51:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:52 smithi110 ceph-mon[35497]: pgmap v905: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-05-17T04:51:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:52 smithi050 ceph-mon[31739]: pgmap v905: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-05-17T04:51:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 775.0 s, 99.0 tps, lat 1019.560 ms stddev 240.376 2024-05-17T04:51:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:54 smithi105 ceph-mon[34795]: pgmap v906: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-17T04:51:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:54 smithi110 ceph-mon[35497]: pgmap v906: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-17T04:51:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:54 smithi050 ceph-mon[31739]: pgmap v906: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-17T04:51:55.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:55 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:55.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:55 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:55.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:55 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:51:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:56 smithi105 ceph-mon[34795]: pgmap v907: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:56 smithi110 ceph-mon[35497]: pgmap v907: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:56 smithi050 ceph-mon[31739]: pgmap v907: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-05-17T04:51:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 780.0 s, 94.0 tps, lat 1051.186 ms stddev 237.318 2024-05-17T04:51:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:51:58 smithi105 ceph-mon[34795]: pgmap v908: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-05-17T04:51:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:51:58 smithi110 ceph-mon[35497]: pgmap v908: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-05-17T04:51:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:51:58 smithi050 ceph-mon[31739]: pgmap v908: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-05-17T04:52:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:00 smithi105 ceph-mon[34795]: pgmap v909: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-05-17T04:52:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:00 smithi110 ceph-mon[35497]: pgmap v909: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-05-17T04:52:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:00 smithi050 ceph-mon[31739]: pgmap v909: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.1 MiB/s wr, 323 op/s 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.679+0000 7f6628474640 1 -- 172.21.15.50:0/3434864515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620103b00 msgr2=0x7f662010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.679+0000 7f6628474640 1 --2- 172.21.15.50:0/3434864515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620103b00 0x7f662010a400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6618009ca0 tx=0x7f661802a8c0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 -- 172.21.15.50:0/3434864515 shutdown_connections 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 --2- 172.21.15.50:0/3434864515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620103b00 0x7f662010a400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 --2- 172.21.15.50:0/3434864515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f66201031b0 0x7f66201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 --2- 172.21.15.50:0/3434864515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6620102090 0x7f66201024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 -- 172.21.15.50:0/3434864515 >> 172.21.15.50:0/3434864515 conn(0x7f66200fd8b0 msgr2=0x7f66200ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 -- 172.21.15.50:0/3434864515 shutdown_connections 2024-05-17T04:52:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.680+0000 7f6628474640 1 -- 172.21.15.50:0/3434864515 wait complete. 2024-05-17T04:52:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.681+0000 7f6628474640 1 Processor -- start 2024-05-17T04:52:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.681+0000 7f6628474640 1 -- start start 2024-05-17T04:52:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620102090 0x7f6620198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6620103b00 0x7f662019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f662010da20 con 0x7f6620103b00 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f662010d8a0 con 0x7f66201031b0 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6628474640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f662010dba0 con 0x7f6620102090 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38372/0 (socket says 172.21.15.50:38372) 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 -- 172.21.15.50:0/994663728 learned_addr learned my addr 172.21.15.50:0/994663728 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6626a11640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6620103b00 0x7f662019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6626210640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620102090 0x7f6620198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 -- 172.21.15.50:0/994663728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620102090 msgr2=0x7f6620198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620102090 0x7f6620198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 -- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6620103b00 msgr2=0x7f662019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6620103b00 0x7f662019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.682+0000 7f6625a0f640 1 -- 172.21.15.50:0/994663728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66201a0260 con 0x7f66201031b0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f6625a0f640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661000e9f0 tx=0x7f661000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f66177fe640 1 -- 172.21.15.50:0/994663728 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661001b070 con 0x7f66201031b0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f66177fe640 1 -- 172.21.15.50:0/994663728 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6610016410 con 0x7f66201031b0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f66177fe640 1 -- 172.21.15.50:0/994663728 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610015610 con 0x7f66201031b0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662006af80 con 0x7f66201031b0 2024-05-17T04:52:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.683+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662006b420 con 0x7f66201031b0 2024-05-17T04:52:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.687+0000 7f662f408740 1 Processor -- start 2024-05-17T04:52:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 -- start start 2024-05-17T04:52:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9ffedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ff5fcc0 con 0x55ef9fc212f0 2024-05-17T04:52:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ff4f860 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f662f408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ff538a0 con 0x55ef9fc294d0 2024-05-17T04:52:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f66177fe640 1 -- 172.21.15.50:0/994663728 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f661001e070 con 0x7f66201031b0 2024-05-17T04:52:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f6626a11640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f6626a11640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:38384/0 (socket says 172.21.15.50:38384) 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.688+0000 7f6626a11640 1 -- 172.21.15.50:0/3843872171 learned_addr learned my addr 172.21.15.50:0/3843872171 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6626a11640 1 -- 172.21.15.50:0/3843872171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 msgr2=0x55ef9fff4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6626a11640 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6626a11640 1 -- 172.21.15.50:0/3843872171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 msgr2=0x55ef9ffedcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6626a11640 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9ffedcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6626a11640 1 -- 172.21.15.50:0/3843872171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9ff17d50 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.689+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.690+0000 7f6626a11640 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6618004ac0 tx=0x7f6618009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc48d423f19f30da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.690+0000 7f6614bfe640 1 -- 172.21.15.50:0/3843872171 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661804a070 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.690+0000 7f6614bfe640 1 -- 172.21.15.50:0/3843872171 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6618036040 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.690+0000 7f6614bfe640 1 -- 172.21.15.50:0/3843872171 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6618046a50 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.690+0000 7f66177fe640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 0x7f660c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f66177fe640 1 -- 172.21.15.50:0/994663728 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f66100e1d60 con 0x7f66201031b0 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f6626210640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 0x7f660c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 -- 172.21.15.50:0/3843872171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 msgr2=0x55ef9ffed770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6618004ac0 tx=0x7f6618009800 comp rx=0 tx=0).stop 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 -- 172.21.15.50:0/3843872171 shutdown_connections 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9ffedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 --2- 172.21.15.50:0/3843872171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9ffed770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.691+0000 7f662f408740 1 -- 172.21.15.50:0/3843872171 >> 172.21.15.50:0/3843872171 conn(0x55ef9fc3e300 msgr2=0x55ef9ffe91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.692+0000 7f662f408740 1 -- 172.21.15.50:0/3843872171 shutdown_connections 2024-05-17T04:52:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.692+0000 7f662f408740 1 -- 172.21.15.50:0/3843872171 wait complete. 2024-05-17T04:52:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.692+0000 7f6626210640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 0x7f660c0b4df0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55ef9ffeeb70 tx=0x7f661c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.692+0000 7f662f408740 1 -- start start 2024-05-17T04:52:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.692+0000 7f662f408740 1 Processor -- start 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fffe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ef9fe8bab0 con 0x55ef9fc212f0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef9ff0c4d0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ef9fff5630 con 0x55ef9fc294d0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f662f408740 10 client.0 _ll_register_callbacks cb 0x55ef9ff4a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f6625a0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f6625a0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:38174/0 (socket says 172.21.15.50:38174) 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f6625a0f640 1 -- 172.21.15.50:0/2480970038 learned_addr learned my addr 172.21.15.50:0/2480970038 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f6626210640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.693+0000 7f6626a11640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fffe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 -- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 msgr2=0x55ef9fffe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fffe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 -- 172.21.15.50:0/2480970038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 msgr2=0x55ef9fff70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 -- 172.21.15.50:0/2480970038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9ff0d340 con 0x55ef9fc294d0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6626a11640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fffe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6625a0f640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66100a5010 tx=0x7f66100a89d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe42080249336f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66100a8f90 con 0x55ef9fc294d0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6618036040 con 0x55ef9fc294d0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f66100a5220 con 0x55ef9fc294d0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f66100ad340 con 0x55ef9fc294d0 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f662f408740 10 client.26254 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.694+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ef9fcb5790 con 0x55ef9fc294d0 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.695+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6618036040 con 0x55ef9fc294d0 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f66100a4900 con 0x55ef9fc294d0 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 10 client.26254 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 20 client.26254 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fc08d80 0x55efa0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 4 client.26254 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9ff61c10 con 0x55ef9fc08d80 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.696+0000 7f6626210640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fc08d80 0x55efa0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.697+0000 7f6609ffb640 10 client.26254 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.697+0000 7f6626210640 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fc08d80 0x55efa0004840 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe4265a97972ddd server_cookie=a05591d6225f5b83 in_seq=0 out_seq=0 2024-05-17T04:52:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.697+0000 7f6609ffb640 1 -- 172.21.15.50:0/2480970038 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef9ff61c10 con 0x55ef9fc08d80 2024-05-17T04:52:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.697+0000 7f6609ffb640 10 client.26254 handle_command_reply: tid=0 2024-05-17T04:52:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 client.26254 shutdown 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 msgr2=0x55ef9fff75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66100a5010 tx=0x7f66100a89d0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 shutdown_connections 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fffe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff75e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fc08d80 0x55efa0004840 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 --2- 172.21.15.50:0/2480970038 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 >> 172.21.15.50:0/2480970038 conn(0x55ef9fc3e300 msgr2=0x55ef9ffe81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f6609ffb640 0 client.26254 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 shutdown_connections 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 1 -- 172.21.15.50:0/2480970038 wait complete. 2024-05-17T04:52:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.698+0000 7f662f408740 20 client.26254 trim_cache size 0 max 16384 2024-05-17T04:52:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.716+0000 7f662f408740 1 Processor -- start 2024-05-17T04:52:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 -- start start 2024-05-17T04:52:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc212f0 0x55ef9ffec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66201ac5f0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66201a9ec0 con 0x55ef9fc212f0 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f662f408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66201a7d10 con 0x55ef9fc294d0 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f6626a11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f6626a11640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43402/0 (socket says 172.21.15.50:43402) 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f6626a11640 1 -- 172.21.15.50:0/3023663824 learned_addr learned my addr 172.21.15.50:0/3023663824 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.717+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 -- 172.21.15.50:0/3023663824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc212f0 msgr2=0x55ef9ffec5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc212f0 0x55ef9ffec5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 -- 172.21.15.50:0/3023663824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 msgr2=0x55ef9fff1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 -- 172.21.15.50:0/3023663824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9fd2e620 con 0x55ef9fc294d0 2024-05-17T04:52:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6626a11640 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f6625a0f640 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff0bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66100a8f20 tx=0x7f661000d850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=264925aeff6e1c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f660bfff640 1 -- 172.21.15.50:0/3023663824 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610088070 con 0x55ef9fc294d0 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f660bfff640 1 -- 172.21.15.50:0/3023663824 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66100b1df0 con 0x55ef9fc294d0 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.718+0000 7f660bfff640 1 -- 172.21.15.50:0/3023663824 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66100a4900 con 0x55ef9fc294d0 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 -- 172.21.15.50:0/3023663824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 msgr2=0x55ef9fff0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff0bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66100a8f20 tx=0x7f661000d850 comp rx=0 tx=0).stop 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 -- 172.21.15.50:0/3023663824 shutdown_connections 2024-05-17T04:52:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc294d0 0x55ef9fff0bd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 --2- 172.21.15.50:0/3023663824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc212f0 0x55ef9ffec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.719+0000 7f662f408740 1 -- 172.21.15.50:0/3023663824 >> 172.21.15.50:0/3023663824 conn(0x55ef9fc3e300 msgr2=0x55ef9ffebed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.721+0000 7f662f408740 1 -- 172.21.15.50:0/3023663824 shutdown_connections 2024-05-17T04:52:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.721+0000 7f662f408740 1 -- 172.21.15.50:0/3023663824 wait complete. 2024-05-17T04:52:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.722+0000 7f662f408740 1 -- start start 2024-05-17T04:52:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.722+0000 7f662f408740 1 Processor -- start 2024-05-17T04:52:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc294d0 0x55ef9fffdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f66201a8380 con 0x55ef9fc212f0 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66201a97f0 con 0x55ef9fc294d0 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66201a8a50 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f662f408740 10 client.0 _ll_register_callbacks cb 0x7f66100a8df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6625a0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc294d0 0x55ef9fffdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:38190/0 (socket says 172.21.15.50:38190) 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 -- 172.21.15.50:0/2978415994 learned_addr learned my addr 172.21.15.50:0/2978415994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626210640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 -- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc294d0 msgr2=0x55ef9fffdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc294d0 0x55ef9fffdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 -- 172.21.15.50:0/2978415994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 msgr2=0x55ef9fff8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.723+0000 7f6626a11640 1 -- 172.21.15.50:0/2978415994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efa006abe0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.724+0000 7f6626a11640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6618009620 tx=0x7f6618046a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2f695d3ca32465 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.724+0000 7f6609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.724+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661804a040 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.724+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66100b1df0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.724+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f66180305c0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.725+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f66180307c0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.725+0000 7f662f408740 10 client.26260 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.725+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55efa006be50 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.725+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f66100b1df0 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f661805aa00 con 0x55ef9fc4a1b0 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 10 client.26260 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 10 client.26260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 20 client.26260 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fbde1c0 0x55efa006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 4 client.26260 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef9ff5b3f0 con 0x55ef9fbde1c0 2024-05-17T04:52:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f6626210640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fbde1c0 0x55efa006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.726+0000 7f6609ffb640 10 client.26260 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.727+0000 7f6626210640 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fbde1c0 0x55efa006e700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81787e621582fead server_cookie=bedca2c993173d5c in_seq=0 out_seq=0 2024-05-17T04:52:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.745+0000 7f6609ffb640 1 -- 172.21.15.50:0/2978415994 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef9ff5b3f0 con 0x55ef9fbde1c0 2024-05-17T04:52:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.745+0000 7f6609ffb640 10 client.26260 handle_command_reply: tid=0 2024-05-17T04:52:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.745+0000 7f662f408740 1 client.26260 shutdown 2024-05-17T04:52:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.746+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 msgr2=0x55ef9fff8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.746+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6618009620 tx=0x7f6618046a80 comp rx=0 tx=0).stop 2024-05-17T04:52:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 shutdown_connections 2024-05-17T04:52:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ef9fc4a1b0 0x55ef9fff8c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef9fc294d0 0x55ef9fffdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ef9fbde1c0 0x55efa006e700 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 --2- 172.21.15.50:0/2978415994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ef9fc212f0 0x55ef9fff8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 >> 172.21.15.50:0/2978415994 conn(0x55ef9fc3e300 msgr2=0x55ef9ffebed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f6609ffb640 0 client.26260 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 shutdown_connections 2024-05-17T04:52:01.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 1 -- 172.21.15.50:0/2978415994 wait complete. 2024-05-17T04:52:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.748+0000 7f662f408740 20 client.26260 trim_cache size 0 max 16384 2024-05-17T04:52:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:52:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:52:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.749+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 msgr2=0x7f660c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.749+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 0x7f660c0b4df0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55ef9ffeeb70 tx=0x7f661c005f90 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.749+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 msgr2=0x7f6620198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.749+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661000e9f0 tx=0x7f661000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 shutdown_connections 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f660c0b28e0 0x7f660c0b4df0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6620103b00 0x7f662019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66201031b0 0x7f6620198c60 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661000e9f0 tx=0x7f661000eeb0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 --2- 172.21.15.50:0/994663728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6620102090 0x7f6620198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.750+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 >> 172.21.15.50:0/994663728 conn(0x7f66200fd8b0 msgr2=0x7f6620106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.752+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 shutdown_connections 2024-05-17T04:52:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:01.752+0000 7f6628474640 1 -- 172.21.15.50:0/994663728 wait complete. 2024-05-17T04:52:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:02 smithi105 ceph-mon[34795]: pgmap v910: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-17T04:52:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:02 smithi110 ceph-mon[35497]: pgmap v910: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-17T04:52:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:02 smithi050 ceph-mon[31739]: pgmap v910: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-17T04:52:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 785.0 s, 105.0 tps, lat 950.213 ms stddev 396.583 2024-05-17T04:52:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:04 smithi105 ceph-mon[34795]: pgmap v911: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-05-17T04:52:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:04 smithi110 ceph-mon[35497]: pgmap v911: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-05-17T04:52:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:04 smithi050 ceph-mon[31739]: pgmap v911: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-05-17T04:52:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:06 smithi105 ceph-mon[34795]: pgmap v912: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-05-17T04:52:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:06 smithi110 ceph-mon[35497]: pgmap v912: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-05-17T04:52:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:06 smithi050 ceph-mon[31739]: pgmap v912: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.4 MiB/s wr, 330 op/s 2024-05-17T04:52:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 790.0 s, 90.2 tps, lat 1133.552 ms stddev 281.510 2024-05-17T04:52:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:08 smithi105 ceph-mon[34795]: pgmap v913: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:52:08.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:08 smithi110 ceph-mon[35497]: pgmap v913: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:52:08.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:08 smithi050 ceph-mon[31739]: pgmap v913: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 2.3 MiB/s wr, 295 op/s 2024-05-17T04:52:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:10 smithi105 ceph-mon[34795]: pgmap v914: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-05-17T04:52:10.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:10 smithi110 ceph-mon[35497]: pgmap v914: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-05-17T04:52:10.577 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:10 smithi050 ceph-mon[31739]: pgmap v914: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.9 MiB/s wr, 284 op/s 2024-05-17T04:52:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:12 smithi105 ceph-mon[34795]: pgmap v915: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-17T04:52:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:12 smithi110 ceph-mon[35497]: pgmap v915: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-17T04:52:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:12 smithi050 ceph-mon[31739]: pgmap v915: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.3 MiB/s wr, 293 op/s 2024-05-17T04:52:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 795.0 s, 111.2 tps, lat 912.415 ms stddev 224.511 2024-05-17T04:52:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:14 smithi105 ceph-mon[34795]: pgmap v916: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-05-17T04:52:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:14 smithi110 ceph-mon[35497]: pgmap v916: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-05-17T04:52:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:14 smithi050 ceph-mon[31739]: pgmap v916: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.878+0000 7fe960074640 1 -- 172.21.15.50:0/666907090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958102080 msgr2=0x7fe958102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.878+0000 7fe960074640 1 --2- 172.21.15.50:0/666907090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958102080 0x7fe958102490 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe950009ca0 tx=0x7fe95002a8c0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 -- 172.21.15.50:0/666907090 shutdown_connections 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 --2- 172.21.15.50:0/666907090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958103af0 0x7fe95810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 --2- 172.21.15.50:0/666907090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe9581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 --2- 172.21.15.50:0/666907090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958102080 0x7fe958102490 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 -- 172.21.15.50:0/666907090 >> 172.21.15.50:0/666907090 conn(0x7fe9580fd880 msgr2=0x7fe9580ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 -- 172.21.15.50:0/666907090 shutdown_connections 2024-05-17T04:52:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.879+0000 7fe960074640 1 -- 172.21.15.50:0/666907090 wait complete. 2024-05-17T04:52:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 Processor -- start 2024-05-17T04:52:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 -- start start 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 0x7fe9581997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958103af0 0x7fe9581a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe95810da10 con 0x7fe958103af0 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe95810d890 con 0x7fe9581031a0 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe960074640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe95810db90 con 0x7fe958102080 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe95e611640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958103af0 0x7fe9581a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe95de10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 0x7fe9581997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34396/0 (socket says 172.21.15.50:34396) 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.880+0000 7fe95d60f640 1 -- 172.21.15.50:0/2319218119 learned_addr learned my addr 172.21.15.50:0/2319218119 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 -- 172.21.15.50:0/2319218119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 msgr2=0x7fe958199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 -- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958103af0 msgr2=0x7fe9581a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958103af0 0x7fe9581a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 -- 172.21.15.50:0/2319218119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9581a1320 con 0x7fe958102080 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95d60f640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe95de10640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 0x7fe9581997e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe95002ade0 tx=0x7fe950003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.881+0000 7fe94affd640 1 -- 172.21.15.50:0/2319218119 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950004450 con 0x7fe958102080 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.882+0000 7fe94affd640 1 -- 172.21.15.50:0/2319218119 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe950040030 con 0x7fe958102080 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.882+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9581a15b0 con 0x7fe958102080 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.882+0000 7fe94affd640 1 -- 172.21.15.50:0/2319218119 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950030cb0 con 0x7fe958102080 2024-05-17T04:52:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.882+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9581a1b50 con 0x7fe958102080 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.885+0000 7fe96700a740 1 Processor -- start 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 -- start start 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe94affd640 1 -- 172.21.15.50:0/2319218119 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe95003b430 con 0x7fe958102080 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255b22f0 0x55f12597ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125985c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe95e611640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1258f0cc0 con 0x55f1255ba4d0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f1258e0860 con 0x55f1255db1b0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe96700a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1258e48a0 con 0x55f1255b22f0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125985c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.886+0000 7fe95e611640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34398/0 (socket says 172.21.15.50:34398) 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 -- 172.21.15.50:0/1482622745 learned_addr learned my addr 172.21.15.50:0/1482622745 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95de10640 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255b22f0 0x55f12597ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 -- 172.21.15.50:0/1482622745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255b22f0 msgr2=0x55f12597ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255b22f0 0x55f12597ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 -- 172.21.15.50:0/1482622745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 msgr2=0x55f125985c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125985c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.887+0000 7fe95e611640 1 -- 172.21.15.50:0/1482622745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1258a8d50 con 0x55f1255db1b0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe95e611640 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098b0 tx=0x7fe954009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7751dcc76588a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe94affd640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 0x7fe9440b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe943fff640 1 -- 172.21.15.50:0/1482622745 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954025070 con 0x55f1255db1b0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe95de10640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 0x7fe9440b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe943fff640 1 -- 172.21.15.50:0/1482622745 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95400eda0 con 0x55f1255db1b0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe943fff640 1 -- 172.21.15.50:0/1482622745 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95401f370 con 0x55f1255db1b0 2024-05-17T04:52:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.888+0000 7fe94affd640 1 -- 172.21.15.50:0/2319218119 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe950103680 con 0x7fe958102080 2024-05-17T04:52:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 -- 172.21.15.50:0/1482622745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 msgr2=0x55f12597e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9540098b0 tx=0x7fe954009d70 comp rx=0 tx=0).stop 2024-05-17T04:52:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe95de10640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 0x7fe9440b4d70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f12597fb70 tx=0x7fe950109550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 -- 172.21.15.50:0/1482622745 shutdown_connections 2024-05-17T04:52:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125985c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255b22f0 0x55f12597ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 --2- 172.21.15.50:0/1482622745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255db1b0 0x55f12597e770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.889+0000 7fe96700a740 1 -- 172.21.15.50:0/1482622745 >> 172.21.15.50:0/1482622745 conn(0x55f1255cf300 msgr2=0x55f12597a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.890+0000 7fe96700a740 1 -- 172.21.15.50:0/1482622745 shutdown_connections 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.890+0000 7fe96700a740 1 -- 172.21.15.50:0/1482622745 wait complete. 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.890+0000 7fe96700a740 1 -- start start 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.890+0000 7fe96700a740 1 Processor -- start 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f1259839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255db1b0 0x55f125984e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f12581cab0 con 0x55f1255ba4d0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f12589d4d0 con 0x55f1255b22f0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1259865d0 con 0x55f1255db1b0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe95e611640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f1259839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe96700a740 10 client.0 _ll_register_callbacks cb 0x55f1258db950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45462/0 (socket says 172.21.15.50:45462) 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.891+0000 7fe95e611640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f1259839f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34412/0 (socket says 172.21.15.50:34412) 2024-05-17T04:52:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 -- 172.21.15.50:0/3045559051 learned_addr learned my addr 172.21.15.50:0/3045559051 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95e611640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255db1b0 0x55f125984e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 -- 172.21.15.50:0/3045559051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255db1b0 msgr2=0x55f125984e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255db1b0 0x55f125984e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 -- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 msgr2=0x55f1259839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f1259839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 -- 172.21.15.50:0/3045559051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f12589e340 con 0x55f1255ba4d0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe95d60f640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe94c012d20 tx=0x7fe94c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed99126202cc50e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.892+0000 7fe9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c00fe20 con 0x55f1255ba4d0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe94c004050 con 0x55f1255ba4d0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95400eda0 con 0x55f1255ba4d0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe94c025050 con 0x55f1255ba4d0 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe96700a740 10 client.16542 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f125646790 con 0x55f1255ba4d0 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.893+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95400eda0 con 0x55f1255ba4d0 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe94c010430 con 0x55f1255ba4d0 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 10 client.16542 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 20 client.16542 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f125599d80 0x55f1259958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 4 client.16542 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1258f2c10 con 0x55f125599d80 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.894+0000 7fe95e611640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f125599d80 0x55f1259958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.895+0000 7fe9417fa640 10 client.16542 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.895+0000 7fe95e611640 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f125599d80 0x55f1259958f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3715e640215cddf server_cookie=2b894730551d157a in_seq=0 out_seq=0 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe9417fa640 1 -- 172.21.15.50:0/3045559051 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1258f2c10 con 0x55f125599d80 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe9417fa640 10 client.16542 handle_command_reply: tid=0 2024-05-17T04:52:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 client.16542 shutdown 2024-05-17T04:52:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 msgr2=0x55f125983f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe94c012d20 tx=0x7fe94c00f790 comp rx=0 tx=0).stop 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 shutdown_connections 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255db1b0 0x55f125984e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f125599d80 0x55f1259958f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1255ba4d0 0x55f125983f30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe9417fa640 0 client.16542 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.896+0000 7fe96700a740 1 --2- 172.21.15.50:0/3045559051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f1259839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.897+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 >> 172.21.15.50:0/3045559051 conn(0x55f1255cf300 msgr2=0x55f1259791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.897+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 shutdown_connections 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.897+0000 7fe96700a740 1 -- 172.21.15.50:0/3045559051 wait complete. 2024-05-17T04:52:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.897+0000 7fe96700a740 20 client.16542 trim_cache size 0 max 16384 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.919+0000 7fe96700a740 1 Processor -- start 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.919+0000 7fe96700a740 1 -- start start 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12597cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12597d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581ac640 con 0x55f125599d80 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581a9f10 con 0x55f1255b22f0 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe96700a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581a7d60 con 0x55f1255ba4d0 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45470/0 (socket says 172.21.15.50:45470) 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 -- 172.21.15.50:0/942318864 learned_addr learned my addr 172.21.15.50:0/942318864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12597d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95d60f640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12597cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 -- 172.21.15.50:0/942318864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 msgr2=0x55f12597d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12597d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 -- 172.21.15.50:0/942318864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 msgr2=0x55f12597cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12597cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 -- 172.21.15.50:0/942318864 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1256bf620 con 0x55f125599d80 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95d60f640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12597cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.920+0000 7fe95e611640 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d838700761a7a6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.921+0000 7fe9437fe640 1 -- 172.21.15.50:0/942318864 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954025040 con 0x55f125599d80 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.921+0000 7fe9437fe640 1 -- 172.21.15.50:0/942318864 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95400eda0 con 0x55f125599d80 2024-05-17T04:52:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.921+0000 7fe9437fe640 1 -- 172.21.15.50:0/942318864 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540054a0 con 0x55f125599d80 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 -- 172.21.15.50:0/942318864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 msgr2=0x55f1259826d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c00 tx=0x7fe954005f90 comp rx=0 tx=0).stop 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 -- 172.21.15.50:0/942318864 shutdown_connections 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12597d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12597cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 --2- 172.21.15.50:0/942318864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f1259826d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 -- 172.21.15.50:0/942318864 >> 172.21.15.50:0/942318864 conn(0x55f1255cf300 msgr2=0x55f125981fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 -- 172.21.15.50:0/942318864 shutdown_connections 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.922+0000 7fe96700a740 1 -- 172.21.15.50:0/942318864 wait complete. 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.923+0000 7fe96700a740 1 -- start start 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.923+0000 7fe96700a740 1 Processor -- start 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.923+0000 7fe96700a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f12598d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.923+0000 7fe96700a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe96700a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12598eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe96700a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581a83d0 con 0x55f125599d80 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe96700a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581a9840 con 0x55f1255b22f0 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe96700a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9581a8aa0 con 0x55f1255ba4d0 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe96700a740 10 client.0 _ll_register_callbacks cb 0x7fe94c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34416/0 (socket says 172.21.15.50:34416) 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95e611640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12598eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 -- 172.21.15.50:0/1141428700 learned_addr learned my addr 172.21.15.50:0/1141428700 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95e611640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f12598d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 -- 172.21.15.50:0/1141428700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 msgr2=0x55f12598eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12598eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 -- 172.21.15.50:0/1141428700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 msgr2=0x55f12598d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f12598d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 -- 172.21.15.50:0/1141428700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f12598f170 con 0x55f1255b22f0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe95d60f640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0059f0 tx=0x7fe94c010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd6fd8257e2d760 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.924+0000 7fe9417fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c000e60 con 0x55f1255b22f0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe94c018040 con 0x55f1255b22f0 2024-05-17T04:52:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95400eda0 con 0x55f1255b22f0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe94c02d070 con 0x55f1255b22f0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe96700a740 10 client.26300 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f1259fdc60 con 0x55f1255b22f0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.925+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe95400eda0 con 0x55f1255b22f0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe94c0037f0 con 0x55f1255b22f0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 10 client.26300 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 20 client.26300 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f1255db1b0 0x55f125a005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 4 client.26300 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1258ec3f0 con 0x55f1255db1b0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.926+0000 7fe95e611640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f1255db1b0 0x55f125a005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.927+0000 7fe9417fa640 10 client.26300 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.927+0000 7fe95e611640 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f1255db1b0 0x55f125a005c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1515d508c45386e2 server_cookie=30555a0ba5c09eb3 in_seq=0 out_seq=0 2024-05-17T04:52:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.956+0000 7fe9417fa640 1 -- 172.21.15.50:0/1141428700 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f1258ec3f0 con 0x55f1255db1b0 2024-05-17T04:52:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.956+0000 7fe9417fa640 10 client.26300 handle_command_reply: tid=0 2024-05-17T04:52:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.956+0000 7fe96700a740 1 client.26300 shutdown 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.956+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 msgr2=0x55f12598dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.956+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0059f0 tx=0x7fe94c010430 comp rx=0 tx=0).stop 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 shutdown_connections 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1255ba4d0 0x55f12598eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f1255b22f0 0x55f12598dc50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55f1255db1b0 0x55f125a005c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 --2- 172.21.15.50:0/1141428700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f125599d80 0x55f12598d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe9417fa640 0 client.26300 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 >> 172.21.15.50:0/1141428700 conn(0x55f1255cf300 msgr2=0x55f1259787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 shutdown_connections 2024-05-17T04:52:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 1 -- 172.21.15.50:0/1141428700 wait complete. 2024-05-17T04:52:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.957+0000 7fe96700a740 20 client.26300 trim_cache size 0 max 16384 2024-05-17T04:52:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:52:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:52:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 msgr2=0x7fe9440b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 0x7fe9440b4d70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f12597fb70 tx=0x7fe950109550 comp rx=0 tx=0).stop 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 msgr2=0x7fe9581997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 0x7fe9581997e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe95002ade0 tx=0x7fe950003c20 comp rx=0 tx=0).stop 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 shutdown_connections 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9440b2860 0x7fe9440b4d70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe958103af0 0x7fe9581a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9581031a0 0x7fe958199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 --2- 172.21.15.50:0/2319218119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe958102080 0x7fe9581997e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 >> 172.21.15.50:0/2319218119 conn(0x7fe9580fd880 msgr2=0x7fe9580ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 shutdown_connections 2024-05-17T04:52:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:14.958+0000 7fe960074640 1 -- 172.21.15.50:0/2319218119 wait complete. 2024-05-17T04:52:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:16 smithi105 ceph-mon[34795]: pgmap v917: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-05-17T04:52:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:16 smithi110 ceph-mon[35497]: pgmap v917: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-05-17T04:52:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:16 smithi050 ceph-mon[31739]: pgmap v917: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 2.5 MiB/s wr, 340 op/s 2024-05-17T04:52:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 800.0 s, 107.6 tps, lat 920.636 ms stddev 220.373 2024-05-17T04:52:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:18 smithi105 ceph-mon[34795]: pgmap v918: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-05-17T04:52:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:18 smithi110 ceph-mon[35497]: pgmap v918: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-05-17T04:52:18.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:18 smithi050 ceph-mon[31739]: pgmap v918: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-05-17T04:52:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:20 smithi105 ceph-mon[34795]: pgmap v919: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-05-17T04:52:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:20 smithi110 ceph-mon[35497]: pgmap v919: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-05-17T04:52:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:20 smithi050 ceph-mon[31739]: pgmap v919: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-05-17T04:52:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:22 smithi105 ceph-mon[34795]: pgmap v920: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 370 op/s 2024-05-17T04:52:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:22 smithi110 ceph-mon[35497]: pgmap v920: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 370 op/s 2024-05-17T04:52:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:22 smithi050 ceph-mon[31739]: pgmap v920: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.7 MiB/s wr, 370 op/s 2024-05-17T04:52:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 805.0 s, 114.6 tps, lat 886.608 ms stddev 215.028 2024-05-17T04:52:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:24 smithi105 ceph-mon[34795]: pgmap v921: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-05-17T04:52:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:24 smithi110 ceph-mon[35497]: pgmap v921: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-05-17T04:52:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:24 smithi050 ceph-mon[31739]: pgmap v921: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.7 MiB/s wr, 399 op/s 2024-05-17T04:52:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:26 smithi105 ceph-mon[34795]: pgmap v922: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:26 smithi110 ceph-mon[35497]: pgmap v922: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:26 smithi050 ceph-mon[31739]: pgmap v922: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 810.0 s, 117.6 tps, lat 813.241 ms stddev 274.361 2024-05-17T04:52:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.120+0000 7f08832af640 1 -- 172.21.15.50:0/3435851242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874095e90 msgr2=0x7f08740962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.120+0000 7f08832af640 1 --2- 172.21.15.50:0/3435851242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874095e90 0x7f08740962a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0878009900 tx=0x7f087802a400 comp rx=0 tx=0).stop 2024-05-17T04:52:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 -- 172.21.15.50:0/3435851242 shutdown_connections 2024-05-17T04:52:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 --2- 172.21.15.50:0/3435851242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874097900 0x7f087409e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 --2- 172.21.15.50:0/3435851242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874096fb0 0x7f08740973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 --2- 172.21.15.50:0/3435851242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874095e90 0x7f08740962a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 -- 172.21.15.50:0/3435851242 >> 172.21.15.50:0/3435851242 conn(0x7f0874091690 msgr2=0x7f0874093b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 -- 172.21.15.50:0/3435851242 shutdown_connections 2024-05-17T04:52:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.121+0000 7f08832af640 1 -- 172.21.15.50:0/3435851242 wait complete. 2024-05-17T04:52:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 Processor -- start 2024-05-17T04:52:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 -- start start 2024-05-17T04:52:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874096fb0 0x7f087413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874097900 0x7f08741449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f08740a1830 con 0x7f0874096fb0 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08740a16b0 con 0x7f0874097900 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.122+0000 7f08832af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f08740a19b0 con 0x7f0874095e90 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:52404/0 (socket says 172.21.15.50:52404) 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 -- 172.21.15.50:0/315981967 learned_addr learned my addr 172.21.15.50:0/315981967 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f0881aac640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874096fb0 0x7f087413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f0882aae640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874097900 0x7f08741449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 -- 172.21.15.50:0/315981967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874097900 msgr2=0x7f08741449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874097900 0x7f08741449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 -- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874096fb0 msgr2=0x7f087413da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874096fb0 0x7f087413da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 -- 172.21.15.50:0/315981967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0874006c40 con 0x7f0874095e90 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.123+0000 7f08822ad640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0878005ab0 tx=0x7f0878004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.124+0000 7f08737fe640 1 -- 172.21.15.50:0/315981967 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087803f030 con 0x7f0874095e90 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.124+0000 7f08737fe640 1 -- 172.21.15.50:0/315981967 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0878004440 con 0x7f0874095e90 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.124+0000 7f08737fe640 1 -- 172.21.15.50:0/315981967 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878035070 con 0x7f0874095e90 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.124+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0874006ed0 con 0x7f0874095e90 2024-05-17T04:52:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.124+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08740073c0 con 0x7f0874095e90 2024-05-17T04:52:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.127+0000 7f08737fe640 1 -- 172.21.15.50:0/315981967 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0878042030 con 0x7f0874095e90 2024-05-17T04:52:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 Processor -- start 2024-05-17T04:52:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 -- start start 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d07a1b0 0x55ad1d41d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d41df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d38fcc0 con 0x55ad1d0512f0 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d37f860 con 0x55ad1d0594d0 2024-05-17T04:52:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.128+0000 7f088b3d4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d3838a0 con 0x55ad1d07a1b0 2024-05-17T04:52:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f0881aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f0881aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40336/0 (socket says 172.21.15.50:40336) 2024-05-17T04:52:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f0881aac640 1 -- 172.21.15.50:0/2055931314 learned_addr learned my addr 172.21.15.50:0/2055931314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f08822ad640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d41df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f0882aae640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d07a1b0 0x55ad1d41d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.129+0000 7f08737fe640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 0x7f08680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 -- 172.21.15.50:0/2055931314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d07a1b0 msgr2=0x55ad1d41d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d07a1b0 0x55ad1d41d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 -- 172.21.15.50:0/2055931314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 msgr2=0x55ad1d424e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 -- 172.21.15.50:0/2055931314 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1d347d50 con 0x55ad1d0512f0 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f0881aac640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f0882aae640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 0x7f08680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08737fe640 1 -- 172.21.15.50:0/315981967 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f08781027b0 con 0x7f0874095e90 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f08822ad640 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d41df00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f08780ffef0 tx=0x7f087804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22926410e7c1c52 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.130+0000 7f0882aae640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 0x7f08680b4e50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f087c068830 tx=0x7f087c069000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.131+0000 7f0870bfe640 1 -- 172.21.15.50:0/2055931314 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780d2070 con 0x55ad1d0512f0 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.131+0000 7f0870bfe640 1 -- 172.21.15.50:0/2055931314 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08780c9600 con 0x55ad1d0512f0 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.131+0000 7f0870bfe640 1 -- 172.21.15.50:0/2055931314 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780307b0 con 0x55ad1d0512f0 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 -- 172.21.15.50:0/2055931314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 msgr2=0x55ad1d41df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d41df00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f08780ffef0 tx=0x7f087804b000 comp rx=0 tx=0).stop 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 -- 172.21.15.50:0/2055931314 shutdown_connections 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d424e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d41df00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 --2- 172.21.15.50:0/2055931314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d07a1b0 0x55ad1d41d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 -- 172.21.15.50:0/2055931314 >> 172.21.15.50:0/2055931314 conn(0x55ad1d06e300 msgr2=0x55ad1d4193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 -- 172.21.15.50:0/2055931314 shutdown_connections 2024-05-17T04:52:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.132+0000 7f088b3d4740 1 -- 172.21.15.50:0/2055931314 wait complete. 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.133+0000 7f088b3d4740 1 -- start start 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.133+0000 7f088b3d4740 1 Processor -- start 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.133+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d0512f0 0x55ad1d42f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.133+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d42fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 0x55ad1d436ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f088b3d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d2bbab0 con 0x55ad1d07a1b0 2024-05-17T04:52:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f088b3d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d33c4d0 con 0x55ad1d0594d0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f088b3d4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad1d425660 con 0x55ad1d0512f0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d42fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0882aae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 0x55ad1d436ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0882aae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 0x55ad1d436ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45160/0 (socket says 172.21.15.50:45160) 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0882aae640 1 -- 172.21.15.50:0/890384063 learned_addr learned my addr 172.21.15.50:0/890384063 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f08822ad640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d0512f0 0x55ad1d42f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f088b3d4740 10 client.0 _ll_register_callbacks cb 0x55ad1d37a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 -- 172.21.15.50:0/890384063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d0512f0 msgr2=0x55ad1d42f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d0512f0 0x55ad1d42f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 -- 172.21.15.50:0/890384063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 msgr2=0x55ad1d436ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 0x55ad1d436ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 -- 172.21.15.50:0/890384063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1d33d340 con 0x55ad1d0594d0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0881aac640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d42fb80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f086c009560 tx=0x7f086c012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147562a1fb67bf77 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.134+0000 7f0865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c004050 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f086c0041f0 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08780c9600 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f086c025050 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f088b3d4740 10 client.26312 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad1d0e5790 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.135+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f086c00fdb0 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f086c003aa0 con 0x55ad1d0594d0 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 10 client.26312 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 20 client.26312 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d038d80 0x55ad1d43ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 4 client.26312 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:52:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08780c9600 con 0x55ad1d038d80 2024-05-17T04:52:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.136+0000 7f08822ad640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d038d80 0x55ad1d43ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.137+0000 7f0865ffb640 10 client.26312 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.137+0000 7f08822ad640 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d038d80 0x55ad1d43ce40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6ae84381c50f28 server_cookie=c44e9966f73b66a0 in_seq=0 out_seq=0 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.138+0000 7f0865ffb640 1 -- 172.21.15.50:0/890384063 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f08780c9600 con 0x55ad1d038d80 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.138+0000 7f0865ffb640 10 client.26312 handle_command_reply: tid=0 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 client.26312 shutdown 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 msgr2=0x55ad1d42fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d42fb80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f086c009560 tx=0x7f086c012b40 comp rx=0 tx=0).stop 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 shutdown_connections 2024-05-17T04:52:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d038d80 0x55ad1d43ce40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f0865ffb640 0 client.26312 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d07a1b0 0x55ad1d436ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0594d0 0x55ad1d42fb80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 --2- 172.21.15.50:0/890384063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d0512f0 0x55ad1d42f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 >> 172.21.15.50:0/890384063 conn(0x55ad1d06e300 msgr2=0x55ad1d41c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 shutdown_connections 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 1 -- 172.21.15.50:0/890384063 wait complete. 2024-05-17T04:52:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.139+0000 7f088b3d4740 20 client.26312 trim_cache size 0 max 16384 2024-05-17T04:52:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 Processor -- start 2024-05-17T04:52:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 -- start start 2024-05-17T04:52:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d038d80 0x55ad1d436a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0874150380 con 0x55ad1d0512f0 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f087414dc50 con 0x55ad1d06e300 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f088b3d4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f087414baa0 con 0x55ad1d038d80 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f08822ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f0882aae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f0882aae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45176/0 (socket says 172.21.15.50:45176) 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f08822ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40348/0 (socket says 172.21.15.50:40348) 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.162+0000 7f0882aae640 1 -- 172.21.15.50:0/476662469 learned_addr learned my addr 172.21.15.50:0/476662469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f0881aac640 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d038d80 0x55ad1d436a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 -- 172.21.15.50:0/476662469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d038d80 msgr2=0x55ad1d436a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d038d80 0x55ad1d436a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 -- 172.21.15.50:0/476662469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 msgr2=0x55ad1d441630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 -- 172.21.15.50:0/476662469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1d15e620 con 0x55ad1d06e300 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f0882aae640 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f08822ad640 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08780ec510 tx=0x7f0878040f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81907d22e3cf23ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f0867fff640 1 -- 172.21.15.50:0/476662469 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780d2040 con 0x55ad1d06e300 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f0867fff640 1 -- 172.21.15.50:0/476662469 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08780c9af0 con 0x55ad1d06e300 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.163+0000 7f0867fff640 1 -- 172.21.15.50:0/476662469 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780c5280 con 0x55ad1d06e300 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.164+0000 7f088b3d4740 1 -- 172.21.15.50:0/476662469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 msgr2=0x55ad1d421da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.164+0000 7f088b3d4740 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08780ec510 tx=0x7f0878040f80 comp rx=0 tx=0).stop 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 -- 172.21.15.50:0/476662469 shutdown_connections 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d0512f0 0x55ad1d441630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d038d80 0x55ad1d436a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- 172.21.15.50:0/476662469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d06e300 0x55ad1d421da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 -- 172.21.15.50:0/476662469 >> 172.21.15.50:0/476662469 conn(0x55ad1d42ef70 msgr2=0x55ad1d41c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 -- 172.21.15.50:0/476662469 shutdown_connections 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 -- 172.21.15.50:0/476662469 wait complete. 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 -- start start 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 Processor -- start 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 0x55ad1d434bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.165+0000 7f088b3d4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d06e300 0x55ad1d435b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f088b3d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f087414c110 con 0x55ad1d038d80 2024-05-17T04:52:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f088b3d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f087414d580 con 0x55ad1d0512f0 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f0881aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 0x55ad1d434bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f088b3d4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f087414c7e0 con 0x55ad1d06e300 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45190/0 (socket says 172.21.15.50:45190) 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 -- 172.21.15.50:0/4055631381 learned_addr learned my addr 172.21.15.50:0/4055631381 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f088b3d4740 10 client.0 _ll_register_callbacks cb 0x7f086c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f0882aae640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d06e300 0x55ad1d435b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 -- 172.21.15.50:0/4055631381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d06e300 msgr2=0x55ad1d435b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d06e300 0x55ad1d435b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 -- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 msgr2=0x55ad1d434bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 0x55ad1d434bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 -- 172.21.15.50:0/4055631381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1d436110 con 0x55ad1d038d80 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f0881aac640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 0x55ad1d434bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f08822ad640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0878107010 tx=0x7f0878036e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b6ad454b7b6f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.166+0000 7f0865ffb640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780dc070 con 0x55ad1d038d80 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f08780303a0 con 0x55ad1d038d80 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08780c9af0 con 0x55ad1d038d80 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0878049040 con 0x55ad1d038d80 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f088b3d4740 10 client.16566 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ad1d4a55b0 con 0x55ad1d038d80 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.167+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08780c9af0 con 0x55ad1d038d80 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f08780c9de0 con 0x55ad1d038d80 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 10 client.16566 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 20 client.16566 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d0594d0 0x55ad1d4a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 4 client.16566 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:52:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad1d391c10 con 0x55ad1d0594d0 2024-05-17T04:52:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f0881aac640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d0594d0 0x55ad1d4a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f0865ffb640 10 client.16566 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.168+0000 7f0881aac640 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d0594d0 0x55ad1d4a7ba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349298cd36920d21 server_cookie=d5d81dd9e7c29adb in_seq=0 out_seq=0 2024-05-17T04:52:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.205+0000 7f0865ffb640 1 -- 172.21.15.50:0/4055631381 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ad1d391c10 con 0x55ad1d0594d0 2024-05-17T04:52:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.205+0000 7f0865ffb640 10 client.16566 handle_command_reply: tid=0 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.205+0000 7f088b3d4740 1 client.16566 shutdown 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.205+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 msgr2=0x55ad1d4346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.205+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0878107010 tx=0x7f0878036e20 comp rx=0 tx=0).stop 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 shutdown_connections 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad1d06e300 0x55ad1d435b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad1d0512f0 0x55ad1d434bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55ad1d0594d0 0x55ad1d4a7ba0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 --2- 172.21.15.50:0/4055631381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad1d038d80 0x55ad1d4346b0 secure :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0878107010 tx=0x7f0878036e20 comp rx=0 tx=0).stop 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 >> 172.21.15.50:0/4055631381 conn(0x55ad1d42ef70 msgr2=0x55ad1d420d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f0865ffb640 0 client.16566 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 shutdown_connections 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 1 -- 172.21.15.50:0/4055631381 wait complete. 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.206+0000 7f088b3d4740 20 client.16566 trim_cache size 0 max 16384 2024-05-17T04:52:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:52:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:52:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:52:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 msgr2=0x7f08680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 0x7f08680b4e50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f087c068830 tx=0x7f087c069000 comp rx=0 tx=0).stop 2024-05-17T04:52:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 msgr2=0x7f087413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0878005ab0 tx=0x7f0878004040 comp rx=0 tx=0).stop 2024-05-17T04:52:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 shutdown_connections 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0874097900 0x7f08741449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f08680b2940 0x7f08680b4e50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0874096fb0 0x7f087413da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 --2- 172.21.15.50:0/315981967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0874095e90 0x7f087413d4f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 >> 172.21.15.50:0/315981967 conn(0x7f0874091690 msgr2=0x7f087409a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.208+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 shutdown_connections 2024-05-17T04:52:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:28.209+0000 7f08832af640 1 -- 172.21.15.50:0/315981967 wait complete. 2024-05-17T04:52:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:28 smithi105 ceph-mon[34795]: pgmap v923: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 347 op/s 2024-05-17T04:52:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:28 smithi110 ceph-mon[35497]: pgmap v923: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 347 op/s 2024-05-17T04:52:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:28 smithi050 ceph-mon[31739]: pgmap v923: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.5 MiB/s wr, 347 op/s 2024-05-17T04:52:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:30 smithi105 ceph-mon[34795]: pgmap v924: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-05-17T04:52:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:30 smithi110 ceph-mon[35497]: pgmap v924: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-05-17T04:52:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:30 smithi050 ceph-mon[31739]: pgmap v924: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 2.6 MiB/s wr, 371 op/s 2024-05-17T04:52:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:32 smithi105 ceph-mon[34795]: pgmap v925: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.6 MiB/s wr, 348 op/s 2024-05-17T04:52:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:32 smithi110 ceph-mon[35497]: pgmap v925: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.6 MiB/s wr, 348 op/s 2024-05-17T04:52:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:32 smithi050 ceph-mon[31739]: pgmap v925: 129 pgs: 129 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 2.6 MiB/s wr, 348 op/s 2024-05-17T04:52:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 815.0 s, 108.4 tps, lat 929.952 ms stddev 230.976 2024-05-17T04:52:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:34 smithi105 ceph-mon[34795]: pgmap v926: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:52:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:34 smithi110 ceph-mon[35497]: pgmap v926: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:52:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:34 smithi050 ceph-mon[31739]: pgmap v926: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.6 MiB/s wr, 379 op/s 2024-05-17T04:52:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:36 smithi105 ceph-mon[34795]: pgmap v927: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.2 MiB/s wr, 329 op/s 2024-05-17T04:52:36.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:36 smithi110 ceph-mon[35497]: pgmap v927: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.2 MiB/s wr, 329 op/s 2024-05-17T04:52:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:36 smithi050 ceph-mon[31739]: pgmap v927: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.2 MiB/s wr, 329 op/s 2024-05-17T04:52:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 820.0 s, 113.4 tps, lat 892.831 ms stddev 212.730 2024-05-17T04:52:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:38 smithi105 ceph-mon[34795]: pgmap v928: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:52:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:38 smithi110 ceph-mon[35497]: pgmap v928: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:52:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:38 smithi050 ceph-mon[31739]: pgmap v928: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.7 MiB/s wr, 346 op/s 2024-05-17T04:52:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:40 smithi105 ceph-mon[34795]: pgmap v929: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2024-05-17T04:52:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:40 smithi110 ceph-mon[35497]: pgmap v929: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2024-05-17T04:52:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:40 smithi050 ceph-mon[31739]: pgmap v929: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 2.1 MiB/s wr, 341 op/s 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3060014395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68102080 msgr2=0x7f4f68102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3060014395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68102080 0x7f4f68102490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009900 tx=0x7f4f5802a410 comp rx=0 tx=0).stop 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3060014395 shutdown_connections 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3060014395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f68103af0 0x7f4f6810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3060014395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f681031a0 0x7f4f681035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3060014395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68102080 0x7f4f68102490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3060014395 >> 172.21.15.50:0/3060014395 conn(0x7f4f680fd880 msgr2=0x7f4f680ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3060014395 shutdown_connections 2024-05-17T04:52:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.358+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3060014395 wait complete. 2024-05-17T04:52:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 Processor -- start 2024-05-17T04:52:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 -- start start 2024-05-17T04:52:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f68102080 0x7f4f6819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f681031a0 0x7f4f6819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810da80 con 0x7f4f681031a0 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810d900 con 0x7f4f68103af0 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f6dfda640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6810dc00 con 0x7f4f68102080 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.359+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52574/0 (socket says 172.21.15.50:52574) 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f68102080 0x7f4f6819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 -- 172.21.15.50:0/3411489757 learned_addr learned my addr 172.21.15.50:0/3411489757 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f681031a0 0x7f4f6819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 -- 172.21.15.50:0/3411489757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f68102080 msgr2=0x7f4f6819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f68102080 0x7f4f6819bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 -- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f681031a0 msgr2=0x7f4f6819c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f681031a0 0x7f4f6819c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 -- 172.21.15.50:0/3411489757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f681a35e0 con 0x7f4f68103af0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.360+0000 7f4f67fff640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000efd0 tx=0x7f4f6000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.361+0000 7f4f64ff9640 1 -- 172.21.15.50:0/3411489757 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60003b70 con 0x7f4f68103af0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.361+0000 7f4f64ff9640 1 -- 172.21.15.50:0/3411489757 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f60003d10 con 0x7f4f68103af0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.361+0000 7f4f64ff9640 1 -- 172.21.15.50:0/3411489757 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60010850 con 0x7f4f68103af0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.361+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f681a38d0 con 0x7f4f68103af0 2024-05-17T04:52:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.361+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681a3dd0 con 0x7f4f68103af0 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.365+0000 7f4f64ff9640 1 -- 172.21.15.50:0/3411489757 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4f600109f0 con 0x7f4f68103af0 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.365+0000 7f4f74f68740 1 Processor -- start 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.365+0000 7f4f74f68740 1 -- start start 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d8af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d92420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55db28cf4cc0 con 0x55db289b62f0 2024-05-17T04:52:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db28ce4860 con 0x55db289be4d0 2024-05-17T04:52:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f74f68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55db28ce88a0 con 0x55db289df1b0 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d8af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57574/0 (socket says 172.21.15.50:57574) 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f677fe640 1 -- 172.21.15.50:0/586636520 learned_addr learned my addr 172.21.15.50:0/586636520 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d92420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 -- 172.21.15.50:0/586636520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 msgr2=0x55db28d92420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d92420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 -- 172.21.15.50:0/586636520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 msgr2=0x55db28d8b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f66ffd640 1 -- 172.21.15.50:0/586636520 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db28cacd50 con 0x55db289df1b0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f64ff9640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 0x7f4f500b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.366+0000 7f4f677fe640 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f677fe640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 0x7f4f500b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d8af80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f6819cbb0 tx=0x7f4f5c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d444af5401961481 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f64ff9640 1 -- 172.21.15.50:0/3411489757 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f600e2a10 con 0x7f4f68103af0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f4dffb640 1 -- 172.21.15.50:0/586636520 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c005020 con 0x55db289df1b0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f4dffb640 1 -- 172.21.15.50:0/586636520 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f5c0047a0 con 0x55db289df1b0 2024-05-17T04:52:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.367+0000 7f4f4dffb640 1 -- 172.21.15.50:0/586636520 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c005650 con 0x55db289df1b0 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.368+0000 7f4f74f68740 1 -- 172.21.15.50:0/586636520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 msgr2=0x55db28d8af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.368+0000 7f4f677fe640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 0x7f4f500b4df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55db28d8c380 tx=0x7f4f58031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.368+0000 7f4f74f68740 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d8af80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f6819cbb0 tx=0x7f4f5c004730 comp rx=0 tx=0).stop 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 -- 172.21.15.50:0/586636520 shutdown_connections 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d92420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d8b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 --2- 172.21.15.50:0/586636520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d8af80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 -- 172.21.15.50:0/586636520 >> 172.21.15.50:0/586636520 conn(0x55db289d3300 msgr2=0x55db28d869b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 -- 172.21.15.50:0/586636520 shutdown_connections 2024-05-17T04:52:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 -- 172.21.15.50:0/586636520 wait complete. 2024-05-17T04:52:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 -- start start 2024-05-17T04:52:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.369+0000 7f4f74f68740 1 Processor -- start 2024-05-17T04:52:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55db28c20ab0 con 0x55db289b62f0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db28ca14d0 con 0x55db289be4d0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55db28d92ea0 con 0x55db289df1b0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f74f68740 10 client.0 _ll_register_callbacks cb 0x55db28cdf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52592/0 (socket says 172.21.15.50:52592) 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 -- 172.21.15.50:0/2994267620 learned_addr learned my addr 172.21.15.50:0/2994267620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f67fff640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f677fe640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 -- 172.21.15.50:0/2994267620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 msgr2=0x55db28d9be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 -- 172.21.15.50:0/2994267620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 msgr2=0x55db28d949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.370+0000 7f4f66ffd640 1 -- 172.21.15.50:0/2994267620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db28ca2340 con 0x55db289be4d0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c004e10 tx=0x7f4f5c004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198b2bec19b4d212 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004180 con 0x55db289be4d0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5c0047a0 con 0x55db289be4d0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f5c004320 con 0x55db289be4d0 2024-05-17T04:52:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f5c005000 con 0x55db289be4d0 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f74f68740 10 client.26330 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.371+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db28a4a790 con 0x55db289be4d0 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f5c0047a0 con 0x55db289be4d0 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f4f5c026070 con 0x55db289be4d0 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 10 client.26330 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 20 client.26330 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db2899dd80 0x55db28da1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 4 client.26330 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:52:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db28cf6c10 con 0x55db2899dd80 2024-05-17T04:52:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.372+0000 7f4f677fe640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db2899dd80 0x55db28da1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.373+0000 7f4f46ffd640 10 client.26330 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.373+0000 7f4f677fe640 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db2899dd80 0x55db28da1e70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3254eb6c810549ca server_cookie=1441182eaa311f5e in_seq=0 out_seq=0 2024-05-17T04:52:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f46ffd640 1 -- 172.21.15.50:0/2994267620 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db28cf6c10 con 0x55db2899dd80 2024-05-17T04:52:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f46ffd640 10 client.26330 handle_command_reply: tid=0 2024-05-17T04:52:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 client.26330 shutdown 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 msgr2=0x55db28d94f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c004e10 tx=0x7f4f5c004e40 comp rx=0 tx=0).stop 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 shutdown_connections 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d94f00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db2899dd80 0x55db28da1e70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 --2- 172.21.15.50:0/2994267620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 >> 172.21.15.50:0/2994267620 conn(0x55db289d3300 msgr2=0x55db28d89820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f46ffd640 0 client.26330 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 shutdown_connections 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 1 -- 172.21.15.50:0/2994267620 wait complete. 2024-05-17T04:52:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.374+0000 7f4f74f68740 20 client.26330 trim_cache size 0 max 16384 2024-05-17T04:52:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.395+0000 7f4f74f68740 1 Processor -- start 2024-05-17T04:52:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.395+0000 7f4f74f68740 1 -- start start 2024-05-17T04:52:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.395+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.395+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:52598/0 (socket says 172.21.15.50:52598) 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57592/0 (socket says 172.21.15.50:57592) 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 -- 172.21.15.50:0/3609901662 learned_addr learned my addr 172.21.15.50:0/3609901662 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681ac640 con 0x55db289b62f0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681a9f10 con 0x55db289be4d0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681a7da0 con 0x55db289df1b0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f67fff640 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 -- 172.21.15.50:0/3609901662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 msgr2=0x55db28d9bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 -- 172.21.15.50:0/3609901662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 msgr2=0x55db28d7cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f66ffd640 1 -- 172.21.15.50:0/3609901662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db28ac3620 con 0x55db289be4d0 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.396+0000 7f4f677fe640 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.397+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00fcf0 tx=0x7f4f5c011640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb869b9eb68e7892 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.397+0000 7f4f4d7fa640 1 -- 172.21.15.50:0/3609901662 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00f990 con 0x55db289be4d0 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.397+0000 7f4f4d7fa640 1 -- 172.21.15.50:0/3609901662 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f5c009cc0 con 0x55db289be4d0 2024-05-17T04:52:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.397+0000 7f4f4d7fa640 1 -- 172.21.15.50:0/3609901662 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004540 con 0x55db289be4d0 2024-05-17T04:52:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 msgr2=0x55db28d7d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00fcf0 tx=0x7f4f5c011640 comp rx=0 tx=0).stop 2024-05-17T04:52:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 shutdown_connections 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289df1b0 0x55db28d9bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d7d430 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 --2- 172.21.15.50:0/3609901662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289b62f0 0x55db28d7cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 >> 172.21.15.50:0/3609901662 conn(0x55db289d3300 msgr2=0x55db28d7c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 shutdown_connections 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.398+0000 7f4f74f68740 1 -- 172.21.15.50:0/3609901662 wait complete. 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 -- start start 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 Processor -- start 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 0x55db28e08210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681a83f0 con 0x55db289df1b0 2024-05-17T04:52:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f66ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:48958/0 (socket says 172.21.15.50:48958) 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f677fe640 1 -- 172.21.15.50:0/206668237 learned_addr learned my addr 172.21.15.50:0/206668237 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681a9840 con 0x55db289be4d0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f681a8aa0 con 0x55db289b62f0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.399+0000 7f4f67fff640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 0x55db28e08210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f74f68740 10 client.0 _ll_register_callbacks cb 0x7f4f5c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 -- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 msgr2=0x55db28d99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 -- 172.21.15.50:0/206668237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 msgr2=0x55db28e08210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 0x55db28e08210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db28d9c830 con 0x55db289b62f0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f67fff640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 0x55db28e08210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f677fe640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f580057c0 tx=0x7f4f58009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c69720d4bd84f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58059060 con 0x55db289b62f0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.400+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f58003dc0 con 0x55db289b62f0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5c009cc0 con 0x55db289b62f0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f58060050 con 0x55db289b62f0 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 10 client.26314 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db28e09690 con 0x55db289b62f0 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f58031990 con 0x55db289b62f0 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f4f58044040 con 0x55db289b62f0 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 10 client.26314 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 20 client.26314 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.401+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db289731c0 0x55db28e0c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.402+0000 7f4f74f68740 4 client.26314 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:52:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.402+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4f5c009cc0 con 0x55db289731c0 2024-05-17T04:52:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.402+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db289731c0 0x55db28e0c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.402+0000 7f4f46ffd640 10 client.26314 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.402+0000 7f4f66ffd640 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db289731c0 0x55db28e0c060 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b734b00eeb58f19 server_cookie=f92153a491732cbb in_seq=0 out_seq=0 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.418+0000 7f4f46ffd640 1 -- 172.21.15.50:0/206668237 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f4f5c009cc0 con 0x55db289731c0 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.418+0000 7f4f46ffd640 10 client.26314 handle_command_reply: tid=0 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.418+0000 7f4f74f68740 1 client.26314 shutdown 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 msgr2=0x55db28d99300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f580057c0 tx=0x7f4f58009260 comp rx=0 tx=0).stop 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 shutdown_connections 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55db289731c0 0x55db28e0c060 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db289df1b0 0x55db28e08210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f46ffd640 0 client.26314 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db289be4d0 0x55db28d99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 --2- 172.21.15.50:0/206668237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55db289b62f0 0x55db28d99300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 >> 172.21.15.50:0/206668237 conn(0x55db289d3300 msgr2=0x55db28d7d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 shutdown_connections 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 1 -- 172.21.15.50:0/206668237 wait complete. 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.419+0000 7f4f74f68740 20 client.26314 trim_cache size 0 max 16384 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:52:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:52:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 msgr2=0x7f4f500b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 0x7f4f500b4df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55db28d8c380 tx=0x7f4f58031400 comp rx=0 tx=0).stop 2024-05-17T04:52:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 msgr2=0x7f4f681a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000efd0 tx=0x7f4f6000c6a0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 shutdown_connections 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f68103af0 0x7f4f681a2f70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f500b28e0 0x7f4f500b4df0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f681031a0 0x7f4f6819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 --2- 172.21.15.50:0/3411489757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f68102080 0x7f4f6819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 >> 172.21.15.50:0/3411489757 conn(0x7f4f680fd880 msgr2=0x7f4f680ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 shutdown_connections 2024-05-17T04:52:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:41.420+0000 7f4f6dfda640 1 -- 172.21.15.50:0/3411489757 wait complete. 2024-05-17T04:52:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:42 smithi105 ceph-mon[34795]: pgmap v930: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:52:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:42 smithi110 ceph-mon[35497]: pgmap v930: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:52:42.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:42 smithi050 ceph-mon[31739]: pgmap v930: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 2.5 MiB/s wr, 350 op/s 2024-05-17T04:52:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 825.0 s, 112.4 tps, lat 896.238 ms stddev 215.352 2024-05-17T04:52:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:44 smithi105 ceph-mon[34795]: pgmap v931: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 394 op/s 2024-05-17T04:52:44.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:44 smithi110 ceph-mon[35497]: pgmap v931: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 394 op/s 2024-05-17T04:52:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:44 smithi050 ceph-mon[31739]: pgmap v931: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.8 MiB/s wr, 394 op/s 2024-05-17T04:52:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:46 smithi105 ceph-mon[34795]: pgmap v932: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-05-17T04:52:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:46 smithi110 ceph-mon[35497]: pgmap v932: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-05-17T04:52:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:46 smithi050 ceph-mon[31739]: pgmap v932: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 2.0 MiB/s wr, 323 op/s 2024-05-17T04:52:47.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 830.0 s, 102.6 tps, lat 967.995 ms stddev 243.527 2024-05-17T04:52:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:48 smithi105 ceph-mon[34795]: pgmap v933: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:52:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:48 smithi110 ceph-mon[35497]: pgmap v933: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:52:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:48 smithi050 ceph-mon[31739]: pgmap v933: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:52:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:50 smithi105 ceph-mon[34795]: pgmap v934: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 354 op/s 2024-05-17T04:52:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:50 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:52:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:50 smithi110 ceph-mon[35497]: pgmap v934: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 354 op/s 2024-05-17T04:52:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:50 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:52:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:50 smithi050 ceph-mon[31739]: pgmap v934: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 2.2 MiB/s wr, 354 op/s 2024-05-17T04:52:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:50 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:52:51.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:52:51.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:52:51.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:52:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:52:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:52:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:52:51.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:52:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:52:51.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:52:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:52 smithi105 ceph-mon[34795]: pgmap v935: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:52 smithi110 ceph-mon[35497]: pgmap v935: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:52 smithi050 ceph-mon[31739]: pgmap v935: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-05-17T04:52:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 835.0 s, 116.0 tps, lat 855.164 ms stddev 316.883 2024-05-17T04:52:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:54 smithi105 ceph-mon[34795]: pgmap v936: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-05-17T04:52:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.563+0000 7f7799012640 1 -- 172.21.15.50:0/834081262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794108fa0 msgr2=0x7f779410b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.563+0000 7f7799012640 1 --2- 172.21.15.50:0/834081262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794108fa0 0x7f779410b3c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f778c009ca0 tx=0x7f778c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 -- 172.21.15.50:0/834081262 shutdown_connections 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 --2- 172.21.15.50:0/834081262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794108fa0 0x7f779410b3c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 --2- 172.21.15.50:0/834081262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7794106640 0x7f7794108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 --2- 172.21.15.50:0/834081262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77941014d0 0x7f77941018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 -- 172.21.15.50:0/834081262 >> 172.21.15.50:0/834081262 conn(0x7f77940fd010 msgr2=0x7f77940ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 -- 172.21.15.50:0/834081262 shutdown_connections 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.564+0000 7f7799012640 1 -- 172.21.15.50:0/834081262 wait complete. 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 Processor -- start 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 -- start start 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 0x7f779419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:54 smithi110 ceph-mon[35497]: pgmap v936: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-05-17T04:52:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7794108fa0 0x7f77941a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f779410ebb0 con 0x7f7794106640 2024-05-17T04:52:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f779410ea30 con 0x7f77941014d0 2024-05-17T04:52:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7799012640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f779410ed30 con 0x7f7794108fa0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f779259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 0x7f779419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46572/0 (socket says 172.21.15.50:46572) 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f779259c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 0x7f779419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33280/0 (socket says 172.21.15.50:33280) 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f7792d9d640 1 -- 172.21.15.50:0/3003666868 learned_addr learned my addr 172.21.15.50:0/3003666868 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.565+0000 7f779359e640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7794108fa0 0x7f77941a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 -- 172.21.15.50:0/3003666868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7794108fa0 msgr2=0x7f77941a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7794108fa0 0x7f77941a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 -- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 msgr2=0x7f779419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 0x7f779419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 -- 172.21.15.50:0/3003666868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77941a34d0 con 0x7f77941014d0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7792d9d640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f777bfff640 1 -- 172.21.15.50:0/3003666868 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400ccc0 con 0x7f77941014d0 2024-05-17T04:52:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77941a37c0 con 0x7f77941014d0 2024-05-17T04:52:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f777bfff640 1 -- 172.21.15.50:0/3003666868 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77840054b0 con 0x7f77941014d0 2024-05-17T04:52:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f777bfff640 1 -- 172.21.15.50:0/3003666868 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784010430 con 0x7f77941014d0 2024-05-17T04:52:54.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.566+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77941a3d10 con 0x7f77941014d0 2024-05-17T04:52:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 Processor -- start 2024-05-17T04:52:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 -- start start 2024-05-17T04:52:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f777bfff640 1 -- 172.21.15.50:0/3003666868 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f77840105d0 con 0x7f77941014d0 2024-05-17T04:52:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bebc1b0 0x56541c25f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c25fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c266c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56541c1d1cc0 con 0x56541be932f0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56541c1c1860 con 0x56541be9b4d0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779ffab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56541c1c58a0 con 0x56541bebc1b0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c25fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c266c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c25fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33296/0 (socket says 172.21.15.50:33296) 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f7792d9d640 1 -- 172.21.15.50:0/1270394506 learned_addr learned my addr 172.21.15.50:0/1270394506 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.570+0000 7f779359e640 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bebc1b0 0x56541c25f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 -- 172.21.15.50:0/1270394506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bebc1b0 msgr2=0x56541c25f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bebc1b0 0x56541c25f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 -- 172.21.15.50:0/1270394506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 msgr2=0x56541c25fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c25fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 -- 172.21.15.50:0/1270394506 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541c189d50 con 0x56541be9b4d0 2024-05-17T04:52:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779259c640 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c266c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7788009bf0 tx=0x7f7788011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07541f3a2a7bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f777bfff640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 0x7f777c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f77797fa640 1 -- 172.21.15.50:0/1270394506 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788004050 con 0x56541be9b4d0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f77797fa640 1 -- 172.21.15.50:0/1270394506 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7788005020 con 0x56541be9b4d0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f77797fa640 1 -- 172.21.15.50:0/1270394506 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788004ca0 con 0x56541be9b4d0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.571+0000 7f779359e640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 0x7f777c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f777bfff640 1 -- 172.21.15.50:0/3003666868 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f77840e2930 con 0x7f77941014d0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779359e640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 0x7f777c0b4d70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56541c260960 tx=0x7f778c031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 -- 172.21.15.50:0/1270394506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 msgr2=0x56541c266c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c266c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7788009bf0 tx=0x7f7788011810 comp rx=0 tx=0).stop 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 -- 172.21.15.50:0/1270394506 shutdown_connections 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c266c10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c25fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 --2- 172.21.15.50:0/1270394506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bebc1b0 0x56541c25f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.572+0000 7f779ffab740 1 -- 172.21.15.50:0/1270394506 >> 172.21.15.50:0/1270394506 conn(0x56541beb0300 msgr2=0x56541c25b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 -- 172.21.15.50:0/1270394506 shutdown_connections 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 -- 172.21.15.50:0/1270394506 wait complete. 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 -- start start 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 Processor -- start 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 0x56541c2695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.573+0000 7f779ffab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 0x56541c270500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779ffab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56541c0fdab0 con 0x56541be932f0 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779ffab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56541c17e4d0 con 0x56541bebc1b0 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779ffab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56541c267690 con 0x56541be9b4d0 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33302/0 (socket says 172.21.15.50:33302) 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 -- 172.21.15.50:0/1989592305 learned_addr learned my addr 172.21.15.50:0/1989592305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779ffab740 10 client.0 _ll_register_callbacks cb 0x56541c1bc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779359e640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 0x56541c270500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779259c640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 0x56541c2695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 -- 172.21.15.50:0/1989592305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 msgr2=0x56541c2695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 0x56541c2695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 -- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 msgr2=0x56541c270500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 0x56541c270500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f7792d9d640 1 -- 172.21.15.50:0/1989592305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c17f340 con 0x56541be932f0 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779359e640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 0x56541c270500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.574+0000 7f779259c640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 0x56541c2695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f77840e82d0 tx=0x7f77840a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75965fe9f08e3880 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f7776ff5640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:52:54.579 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:54 smithi050 ceph-mon[31739]: pgmap v936: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.3 MiB/s wr, 371 op/s 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a52f0 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f77840b2530 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7788005020 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f77840a9670 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f779ffab740 10 client.16593 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.575+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56541bf27790 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.576+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7788005020 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f77840c40d0 con 0x56541be932f0 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 10 client.16593 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 20 client.16593 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541be7ad80 0x56541c2767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 4 client.16593 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541c1d3c10 con 0x56541be7ad80 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.577+0000 7f779259c640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541be7ad80 0x56541c2767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.578+0000 7f779259c640 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541be7ad80 0x56541c2767f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bbcae76f1e76d4 server_cookie=1c3a9feaa8c7d98a in_seq=0 out_seq=0 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.578+0000 7f7776ff5640 10 client.16593 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.579+0000 7f7776ff5640 1 -- 172.21.15.50:0/1989592305 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56541c1d3c10 con 0x56541be7ad80 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.579+0000 7f7776ff5640 10 client.16593 handle_command_reply: tid=0 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.579+0000 7f779ffab740 1 client.16593 shutdown 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.579+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 msgr2=0x56541c269060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.579+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f77840e82d0 tx=0x7f77840a5ae0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 shutdown_connections 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541bebc1b0 0x56541c270500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be9b4d0 0x56541c2695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541be7ad80 0x56541c2767f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 --2- 172.21.15.50:0/1989592305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c269060 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 >> 172.21.15.50:0/1989592305 conn(0x56541beb0300 msgr2=0x56541c25a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f7776ff5640 0 client.16593 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 shutdown_connections 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 1 -- 172.21.15.50:0/1989592305 wait complete. 2024-05-17T04:52:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.580+0000 7f779ffab740 20 client.16593 trim_cache size 0 max 16384 2024-05-17T04:52:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 Processor -- start 2024-05-17T04:52:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 -- start start 2024-05-17T04:52:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 0x56541c25b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be9b4d0 0x56541c25ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77941b0a80 con 0x56541be9b4d0 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77941ae350 con 0x56541beb0300 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.607+0000 7f779ffab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77941ac1a0 con 0x56541be932f0 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f779359e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be9b4d0 0x56541c25ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46602/0 (socket says 172.21.15.50:46602) 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f779259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 0x56541c25b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 -- 172.21.15.50:0/263392924 learned_addr learned my addr 172.21.15.50:0/263392924 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 -- 172.21.15.50:0/263392924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 msgr2=0x56541c25b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 0x56541c25b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 -- 172.21.15.50:0/263392924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be9b4d0 msgr2=0x56541c25ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be9b4d0 0x56541c25ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 -- 172.21.15.50:0/263392924 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541bfa0620 con 0x56541beb0300 2024-05-17T04:52:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f779259c640 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 0x56541c25b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:52:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.608+0000 7f7792d9d640 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77840a9af0 tx=0x7f77840a5b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27486c316a77132 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.609+0000 7f7778ff9640 1 -- 172.21.15.50:0/263392924 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a52f0 con 0x56541beb0300 2024-05-17T04:52:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.609+0000 7f7778ff9640 1 -- 172.21.15.50:0/263392924 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77840aa260 con 0x56541beb0300 2024-05-17T04:52:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.609+0000 7f7778ff9640 1 -- 172.21.15.50:0/263392924 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a94b0 con 0x56541beb0300 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 -- 172.21.15.50:0/263392924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 msgr2=0x56541c255f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77840a9af0 tx=0x7f77840a5b50 comp rx=0 tx=0).stop 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 -- 172.21.15.50:0/263392924 shutdown_connections 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be9b4d0 0x56541c25ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be932f0 0x56541c25b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 --2- 172.21.15.50:0/263392924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541beb0300 0x56541c255f00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 -- 172.21.15.50:0/263392924 >> 172.21.15.50:0/263392924 conn(0x56541c25fe20 msgr2=0x56541c25aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 -- 172.21.15.50:0/263392924 shutdown_connections 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.610+0000 7f779ffab740 1 -- 172.21.15.50:0/263392924 wait complete. 2024-05-17T04:52:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.611+0000 7f779ffab740 1 -- start start 2024-05-17T04:52:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.611+0000 7f779ffab740 1 Processor -- start 2024-05-17T04:52:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c26c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c26ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 0x56541c2dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c26ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c26ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:46606/0 (socket says 172.21.15.50:46606) 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779259c640 1 -- 172.21.15.50:0/1962101369 learned_addr learned my addr 172.21.15.50:0/1962101369 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77941ac810 con 0x56541be932f0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77941adc80 con 0x56541be9b4d0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77941acee0 con 0x56541beb0300 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779ffab740 10 client.0 _ll_register_callbacks cb 0x7f77840e8220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f779359e640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 0x56541c2dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.612+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c26c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 -- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 msgr2=0x56541c26ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c26ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 -- 172.21.15.50:0/1962101369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 msgr2=0x56541c26c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c26c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c26fcc0 con 0x56541beb0300 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779359e640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 0x56541c2dcb90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f778c031920 tx=0x7f778c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6739d1306cc3fba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f7776ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c058060 con 0x56541beb0300 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f778c004720 con 0x56541beb0300 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77840aa260 con 0x56541beb0300 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.613+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f778c05f050 con 0x56541beb0300 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 10 client.26332 fetch_fsmap learned FSMap version 43 2024-05-17T04:52:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56541c2ddfc0 con 0x56541beb0300 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f778c036050 con 0x56541beb0300 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f778c049490 con 0x56541beb0300 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 10 client.26332 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 20 client.26332 populate_metadata read hostname 'smithi050' 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541bebc1b0 0x56541c2e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 4 client.26332 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:52:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f77840aa260 con 0x56541bebc1b0 2024-05-17T04:52:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.614+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541bebc1b0 0x56541c2e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:52:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.615+0000 7f7776ff5640 10 client.26332 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:54.618 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.615+0000 7f7792d9d640 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541bebc1b0 0x56541c2e09e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325575bc47f434eb server_cookie=7d0c27f15df7f61 in_seq=0 out_seq=0 2024-05-17T04:52:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.631+0000 7f7776ff5640 1 -- 172.21.15.50:0/1962101369 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f77840aa260 con 0x56541bebc1b0 2024-05-17T04:52:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.631+0000 7f7776ff5640 10 client.26332 handle_command_reply: tid=0 2024-05-17T04:52:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.631+0000 7f779ffab740 1 client.26332 shutdown 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.631+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 msgr2=0x56541c2dcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 0x56541c2dcb90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f778c031920 tx=0x7f778c0095c0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 shutdown_connections 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541beb0300 0x56541c2dcb90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56541be9b4d0 0x56541c26ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56541bebc1b0 0x56541c2e09e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 --2- 172.21.15.50:0/1962101369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56541be932f0 0x56541c26c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f7776ff5640 0 client.26332 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 >> 172.21.15.50:0/1962101369 conn(0x56541c25fe20 msgr2=0x56541c2597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 shutdown_connections 2024-05-17T04:52:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 1 -- 172.21.15.50:0/1962101369 wait complete. 2024-05-17T04:52:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.632+0000 7f779ffab740 20 client.26332 trim_cache size 0 max 16384 2024-05-17T04:52:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:52:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:52:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 msgr2=0x7f777c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 0x7f777c0b4d70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56541c260960 tx=0x7f778c031000 comp rx=0 tx=0).stop 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 msgr2=0x7f779419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 shutdown_connections 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7794108fa0 0x7f77941a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f777c0b2860 0x7f777c0b4d70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7794106640 0x7f779419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 --2- 172.21.15.50:0/3003666868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77941014d0 0x7f779419b990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:52:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 >> 172.21.15.50:0/3003666868 conn(0x7f77940fd010 msgr2=0x7f77941054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:52:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 shutdown_connections 2024-05-17T04:52:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:52:54.633+0000 7f7799012640 1 -- 172.21.15.50:0/3003666868 wait complete. 2024-05-17T04:52:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:56 smithi105 ceph-mon[34795]: pgmap v937: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.6 MiB/s wr, 359 op/s 2024-05-17T04:52:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:56 smithi110 ceph-mon[35497]: pgmap v937: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.6 MiB/s wr, 359 op/s 2024-05-17T04:52:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:56 smithi050 ceph-mon[31739]: pgmap v937: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.6 MiB/s wr, 359 op/s 2024-05-17T04:52:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 840.0 s, 107.2 tps, lat 949.702 ms stddev 241.915 2024-05-17T04:52:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:52:58 smithi105 ceph-mon[34795]: pgmap v938: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-05-17T04:52:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:52:58 smithi110 ceph-mon[35497]: pgmap v938: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-05-17T04:52:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:52:58 smithi050 ceph-mon[31739]: pgmap v938: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 2.6 MiB/s wr, 344 op/s 2024-05-17T04:53:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:00 smithi105 ceph-mon[34795]: pgmap v939: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.6 MiB/s wr, 368 op/s 2024-05-17T04:53:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:00 smithi110 ceph-mon[35497]: pgmap v939: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.6 MiB/s wr, 368 op/s 2024-05-17T04:53:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:00 smithi050 ceph-mon[31739]: pgmap v939: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.6 MiB/s wr, 368 op/s 2024-05-17T04:53:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:02 smithi105 ceph-mon[34795]: pgmap v940: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:53:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:02 smithi110 ceph-mon[35497]: pgmap v940: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:53:02.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:02 smithi050 ceph-mon[31739]: pgmap v940: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 335 op/s 2024-05-17T04:53:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 845.0 s, 103.8 tps, lat 966.352 ms stddev 228.838 2024-05-17T04:53:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:04 smithi105 ceph-mon[34795]: pgmap v941: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.4 MiB/s wr, 364 op/s 2024-05-17T04:53:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:04 smithi110 ceph-mon[35497]: pgmap v941: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.4 MiB/s wr, 364 op/s 2024-05-17T04:53:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:04 smithi050 ceph-mon[31739]: pgmap v941: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 2.4 MiB/s wr, 364 op/s 2024-05-17T04:53:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:06 smithi105 ceph-mon[34795]: pgmap v942: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:53:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:06 smithi110 ceph-mon[35497]: pgmap v942: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:53:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:06 smithi050 ceph-mon[31739]: pgmap v942: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.6 MiB/s wr, 350 op/s 2024-05-17T04:53:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/1536798710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac06b230 msgr2=0x7f3aac105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/1536798710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac06b230 0x7f3aac105ce0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009900 tx=0x7f3a9c02a410 comp rx=0 tx=0).stop 2024-05-17T04:53:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/1536798710 shutdown_connections 2024-05-17T04:53:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/1536798710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac108e20 0x7f3aac10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/1536798710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac1062b0 0x7f3aac108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/1536798710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac06b230 0x7f3aac105ce0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/1536798710 >> 172.21.15.50:0/1536798710 conn(0x7f3aac0fd880 msgr2=0x7f3aac0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/1536798710 shutdown_connections 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/1536798710 wait complete. 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.767+0000 7f3ab1fcb640 1 Processor -- start 2024-05-17T04:53:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 -- start start 2024-05-17T04:53:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac06b230 0x7f3aac199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 0x7f3aac1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10eba0 con 0x7f3aac1062b0 2024-05-17T04:53:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10ea20 con 0x7f3aac06b230 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3ab1fcb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10ed20 con 0x7f3aac108e20 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38458/0 (socket says 172.21.15.50:38458) 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac06b230 0x7f3aac199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 -- 172.21.15.50:0/2424990456 learned_addr learned my addr 172.21.15.50:0/2424990456 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 0x7f3aac1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 -- 172.21.15.50:0/2424990456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 msgr2=0x7f3aac1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 0x7f3aac1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 -- 172.21.15.50:0/2424990456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac06b230 msgr2=0x7f3aac199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac06b230 0x7f3aac199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aaaffd640 1 -- 172.21.15.50:0/2424990456 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac1a1390 con 0x7f3aac1062b0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.768+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 0x7f3aac1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3aa8ff9640 1 -- 172.21.15.50:0/2424990456 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cd80 con 0x7f3aac1062b0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3aa8ff9640 1 -- 172.21.15.50:0/2424990456 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa00054b0 con 0x7f3aac1062b0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac1a1680 con 0x7f3aac1062b0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3aa8ff9640 1 -- 172.21.15.50:0/2424990456 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0010430 con 0x7f3aac1062b0 2024-05-17T04:53:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.769+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac1a1bd0 con 0x7f3aac1062b0 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3aa8ff9640 1 -- 172.21.15.50:0/2424990456 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3aa00105f0 con 0x7f3aac1062b0 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 Processor -- start 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 -- start start 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46461b0 0x5574c49edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c495bcc0 con 0x5574c461d2f0 2024-05-17T04:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574c494b860 con 0x5574c46461b0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574c494f8a0 con 0x5574c46254d0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46461b0 0x5574c49edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38472/0 (socket says 172.21.15.50:38472) 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.772+0000 7f3aab7fe640 1 -- 172.21.15.50:0/2815569859 learned_addr learned my addr 172.21.15.50:0/2815569859 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 -- 172.21.15.50:0/2815569859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 msgr2=0x5574c49f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 -- 172.21.15.50:0/2815569859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46461b0 msgr2=0x5574c49edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46461b0 0x5574c49edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 -- 172.21.15.50:0/2815569859 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c4913d50 con 0x5574c461d2f0 2024-05-17T04:53:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aa8ff9640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 0x7f3a940b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 0x7f3a940b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3aac06b640 tx=0x7f3a9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fe3db275118be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3aa8ff9640 1 -- 172.21.15.50:0/2424990456 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa00e28b0 con 0x7f3aac1062b0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.773+0000 7f3a91ffb640 1 -- 172.21.15.50:0/2815569859 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c035030 con 0x5574c461d2f0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.774+0000 7f3a91ffb640 1 -- 172.21.15.50:0/2815569859 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a9c040070 con 0x5574c461d2f0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.774+0000 7f3a91ffb640 1 -- 172.21.15.50:0/2815569859 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0317f0 con 0x5574c461d2f0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.774+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2815569859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 msgr2=0x5574c49ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.774+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 0x7f3a940b4d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5574c49eed70 tx=0x7f3aa4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.774+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3aac06b640 tx=0x7f3a9c004040 comp rx=0 tx=0).stop 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2815569859 shutdown_connections 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c461d2f0 0x5574c49ee100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2815569859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46461b0 0x5574c49edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2815569859 >> 172.21.15.50:0/2815569859 conn(0x5574c463a300 msgr2=0x5574c49e51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2815569859 shutdown_connections 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2815569859 wait complete. 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 -- start start 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.775+0000 7f3ab8e83740 1 Processor -- start 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 0x5574c49f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c4887ab0 con 0x5574c46461b0 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574c49084d0 con 0x5574c461d2f0 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574c49f5a40 con 0x5574c46254d0 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3ab8e83740 10 client.0 _ll_register_callbacks cb 0x5574c4946950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 0x5574c49f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38488/0 (socket says 172.21.15.50:38488) 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 0x5574c49f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39542/0 (socket says 172.21.15.50:39542) 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 -- 172.21.15.50:0/441315006 learned_addr learned my addr 172.21.15.50:0/441315006 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 -- 172.21.15.50:0/441315006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 msgr2=0x5574c49f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 -- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 msgr2=0x5574c49f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 0x5574c49f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 -- 172.21.15.50:0/441315006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c4909340 con 0x5574c46461b0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3aabfff640 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40127c0 tx=0x7f3aa4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d448c8b89be6faf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3a8affd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:53:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.776+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4011c60 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa4011e00 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c040070 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3aa4029440 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 10 client.16611 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5574c46b1790 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a9c040070 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f3aa4027070 con 0x5574c46461b0 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 10 client.16611 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 20 client.16611 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c4604d80 0x5574c4a04ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 4 client.16611 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c495dc10 con 0x5574c4604d80 2024-05-17T04:53:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.777+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c4604d80 0x5574c4a04ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.778+0000 7f3a8affd640 10 client.16611 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.778+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c4604d80 0x5574c4a04ce0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f1ef472562e4df server_cookie=ade2748d251c974f in_seq=0 out_seq=0 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3a8affd640 1 -- 172.21.15.50:0/441315006 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574c495dc10 con 0x5574c4604d80 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3a8affd640 10 client.16611 handle_command_reply: tid=0 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 client.16611 shutdown 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 msgr2=0x5574c49fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40127c0 tx=0x7f3aa4004ab0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 shutdown_connections 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c4604d80 0x5574c4a04ce0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c46461b0 0x5574c49fe8b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3a8affd640 0 client.16611 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c46254d0 0x5574c49f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/441315006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c461d2f0 0x5574c49f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 >> 172.21.15.50:0/441315006 conn(0x5574c463a300 msgr2=0x5574c49e8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 shutdown_connections 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 1 -- 172.21.15.50:0/441315006 wait complete. 2024-05-17T04:53:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.779+0000 7f3ab8e83740 20 client.16611 trim_cache size 0 max 16384 2024-05-17T04:53:07.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 850.0 s, 104.6 tps, lat 941.134 ms stddev 217.281 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 Processor -- start 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 -- start start 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ac6c0 con 0x5574c4604d80 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1a9f90 con 0x5574c46254d0 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.803+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1a7de0 con 0x5574c461d2f0 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39552/0 (socket says 172.21.15.50:39552) 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 -- 172.21.15.50:0/2397479867 learned_addr learned my addr 172.21.15.50:0/2397479867 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 -- 172.21.15.50:0/2397479867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 msgr2=0x5574c49e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49e85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 -- 172.21.15.50:0/2397479867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 msgr2=0x5574c49f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 -- 172.21.15.50:0/2397479867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c472a620 con 0x5574c46254d0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3aabfff640 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009690 tx=0x7f3aa4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f090079f3d9e8ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.804+0000 7f3a917fa640 1 -- 172.21.15.50:0/2397479867 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa403c070 con 0x5574c46254d0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.805+0000 7f3a917fa640 1 -- 172.21.15.50:0/2397479867 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa401f9e0 con 0x5574c46254d0 2024-05-17T04:53:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.805+0000 7f3a917fa640 1 -- 172.21.15.50:0/2397479867 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4029df0 con 0x5574c46254d0 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2397479867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 msgr2=0x5574c49e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009690 tx=0x7f3aa4005f10 comp rx=0 tx=0).stop 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2397479867 shutdown_connections 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c49e8b20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/2397479867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2397479867 >> 172.21.15.50:0/2397479867 conn(0x5574c463a300 msgr2=0x5574c49e7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2397479867 shutdown_connections 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- 172.21.15.50:0/2397479867 wait complete. 2024-05-17T04:53:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.806+0000 7f3ab8e83740 1 -- start start 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 Processor -- start 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c4a6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1a8450 con 0x5574c4604d80 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1a98c0 con 0x5574c46254d0 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1a8b20 con 0x5574c461d2f0 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3ab8e83740 10 client.0 _ll_register_callbacks cb 0x7f3aa4012710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3aaaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:44974/0 (socket says 172.21.15.50:44974) 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.807+0000 7f3aaaffd640 1 -- 172.21.15.50:0/201486459 learned_addr learned my addr 172.21.15.50:0/201486459 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 -- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 msgr2=0x5574c4a6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aabfff640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c4a6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c4a6b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 -- 172.21.15.50:0/201486459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 msgr2=0x5574c49fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 -- 172.21.15.50:0/201486459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c4a6b7f0 con 0x5574c461d2f0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3aaaffd640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00a9bb0 tx=0x7f3aa0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4791bc56476bed server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3a8affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00a9570 con 0x5574c461d2f0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3aa0005000 con 0x5574c461d2f0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.808+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa401f9e0 con 0x5574c461d2f0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3aa00a53b0 con 0x5574c461d2f0 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 10 client.26350 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5574c49e9910 con 0x5574c461d2f0 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa00b22e0 con 0x5574c461d2f0 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f3aa00a5590 con 0x5574c461d2f0 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 10 client.26350 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 10 client.26350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 20 client.26350 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c46461b0 0x5574c49ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 4 client.26350 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3aa401f9e0 con 0x5574c46461b0 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.809+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c46461b0 0x5574c49ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.810+0000 7f3a8affd640 10 client.26350 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.810+0000 7f3aab7fe640 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c46461b0 0x5574c49ec2d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e465b66c14dffb server_cookie=425096e668849b1a in_seq=0 out_seq=0 2024-05-17T04:53:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3a8affd640 1 -- 172.21.15.50:0/201486459 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f3aa401f9e0 con 0x5574c46461b0 2024-05-17T04:53:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3a8affd640 10 client.26350 handle_command_reply: tid=0 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 client.26350 shutdown 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 msgr2=0x5574c49fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00a9bb0 tx=0x7f3aa0009310 comp rx=0 tx=0).stop 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 shutdown_connections 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574c46254d0 0x5574c4a6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574c461d2f0 0x5574c49fd720 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5574c46461b0 0x5574c49ec2d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 --2- 172.21.15.50:0/201486459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c4604d80 0x5574c49fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 >> 172.21.15.50:0/201486459 conn(0x5574c463a300 msgr2=0x5574c49e8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3a8affd640 0 client.26350 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 shutdown_connections 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.826+0000 7f3ab8e83740 1 -- 172.21.15.50:0/201486459 wait complete. 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.827+0000 7f3ab8e83740 20 client.26350 trim_cache size 0 max 16384 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:53:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:53:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.827+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 msgr2=0x7f3a940b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.827+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 0x7f3a940b4d70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5574c49eed70 tx=0x7f3aa4009500 comp rx=0 tx=0).stop 2024-05-17T04:53:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.827+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 msgr2=0x7f3aac199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.827+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9f0 tx=0x7f3aa000edf0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 shutdown_connections 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3aac108e20 0x7f3aac1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f3a940b2860 0x7f3a940b4d70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3aac1062b0 0x7f3aac199d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 --2- 172.21.15.50:0/2424990456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aac06b230 0x7f3aac199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 >> 172.21.15.50:0/2424990456 conn(0x7f3aac0fd880 msgr2=0x7f3aac1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 shutdown_connections 2024-05-17T04:53:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:07.828+0000 7f3ab1fcb640 1 -- 172.21.15.50:0/2424990456 wait complete. 2024-05-17T04:53:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:08 smithi105 ceph-mon[34795]: pgmap v943: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.1 MiB/s wr, 309 op/s 2024-05-17T04:53:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:08 smithi110 ceph-mon[35497]: pgmap v943: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.1 MiB/s wr, 309 op/s 2024-05-17T04:53:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:08 smithi050 ceph-mon[31739]: pgmap v943: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 2.1 MiB/s wr, 309 op/s 2024-05-17T04:53:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:10 smithi105 ceph-mon[34795]: pgmap v944: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-05-17T04:53:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:10 smithi110 ceph-mon[35497]: pgmap v944: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-05-17T04:53:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:10 smithi050 ceph-mon[31739]: pgmap v944: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-05-17T04:53:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:12 smithi105 ceph-mon[34795]: pgmap v945: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-05-17T04:53:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:12 smithi110 ceph-mon[35497]: pgmap v945: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-05-17T04:53:12.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:12 smithi050 ceph-mon[31739]: pgmap v945: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 2.3 MiB/s wr, 333 op/s 2024-05-17T04:53:12.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 855.0 s, 103.8 tps, lat 957.813 ms stddev 231.961 2024-05-17T04:53:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:14 smithi105 ceph-mon[34795]: pgmap v946: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.7 MiB/s wr, 371 op/s 2024-05-17T04:53:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:14 smithi110 ceph-mon[35497]: pgmap v946: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.7 MiB/s wr, 371 op/s 2024-05-17T04:53:14.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:14 smithi050 ceph-mon[31739]: pgmap v946: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 2.7 MiB/s wr, 371 op/s 2024-05-17T04:53:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:16 smithi105 ceph-mon[34795]: pgmap v947: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-05-17T04:53:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:16 smithi110 ceph-mon[35497]: pgmap v947: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-05-17T04:53:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:16 smithi050 ceph-mon[31739]: pgmap v947: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 2.1 MiB/s wr, 311 op/s 2024-05-17T04:53:17.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 860.0 s, 84.6 tps, lat 1172.279 ms stddev 277.789 2024-05-17T04:53:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:18 smithi105 ceph-mon[34795]: pgmap v948: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-05-17T04:53:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:18 smithi110 ceph-mon[35497]: pgmap v948: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-05-17T04:53:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:18 smithi050 ceph-mon[31739]: pgmap v948: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-05-17T04:53:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:20 smithi105 ceph-mon[34795]: pgmap v949: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-17T04:53:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:20 smithi110 ceph-mon[35497]: pgmap v949: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-17T04:53:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:20 smithi050 ceph-mon[31739]: pgmap v949: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-05-17T04:53:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 -- 172.21.15.50:0/3818998564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c103af0 msgr2=0x7fe50c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 --2- 172.21.15.50:0/3818998564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c103af0 0x7fe50c10a3f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe504009be0 tx=0x7fe50402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 -- 172.21.15.50:0/3818998564 shutdown_connections 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 --2- 172.21.15.50:0/3818998564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c103af0 0x7fe50c10a3f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 --2- 172.21.15.50:0/3818998564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c1031a0 0x7fe50c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 --2- 172.21.15.50:0/3818998564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c102080 0x7fe50c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 -- 172.21.15.50:0/3818998564 >> 172.21.15.50:0/3818998564 conn(0x7fe50c0fd880 msgr2=0x7fe50c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 -- 172.21.15.50:0/3818998564 shutdown_connections 2024-05-17T04:53:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.967+0000 7fe5123b2640 1 -- 172.21.15.50:0/3818998564 wait complete. 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.968+0000 7fe5123b2640 1 Processor -- start 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.968+0000 7fe5123b2640 1 -- start start 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.968+0000 7fe5123b2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 0x7fe50c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.968+0000 7fe5123b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c1031a0 0x7fe50c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe5123b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 0x7fe50c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe5123b2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c10da60 con 0x7fe50c102080 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe5123b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c10d8e0 con 0x7fe50c103af0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe5123b2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c10dbe0 con 0x7fe50c1031a0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe50b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c1031a0 0x7fe50c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 0x7fe50c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 0x7fe50c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 0x7fe50c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40650/0 (socket says 172.21.15.50:40650) 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe50bfff640 1 -- 172.21.15.50:0/2318228772 learned_addr learned my addr 172.21.15.50:0/2318228772 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 -- 172.21.15.50:0/2318228772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c1031a0 msgr2=0x7fe50c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c1031a0 0x7fe50c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 -- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 msgr2=0x7fe50c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 0x7fe50c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 -- 172.21.15.50:0/2318228772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c1a1350 con 0x7fe50c103af0 2024-05-17T04:53:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.969+0000 7fe51094f640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 0x7fe50c1a0ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b30 tx=0x7fe504030be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.970+0000 7fe5097fa640 1 -- 172.21.15.50:0/2318228772 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504041020 con 0x7fe50c103af0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.970+0000 7fe5097fa640 1 -- 172.21.15.50:0/2318228772 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe504037040 con 0x7fe50c103af0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.970+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c1a15e0 con 0x7fe50c103af0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.970+0000 7fe5097fa640 1 -- 172.21.15.50:0/2318228772 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504036070 con 0x7fe50c103af0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.970+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a1af0 con 0x7fe50c103af0 2024-05-17T04:53:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.973+0000 7fe51926c740 1 Processor -- start 2024-05-17T04:53:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.973+0000 7fe5097fa640 1 -- 172.21.15.50:0/2318228772 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe504044020 con 0x7fe50c103af0 2024-05-17T04:53:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 -- start start 2024-05-17T04:53:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cb031b0 0x56363cea6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cae24d0 0x56363ceadc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56363ce18cc0 con 0x56363cada2f0 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56363ce08860 con 0x56363cb031b0 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe51926c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56363ce0c8a0 con 0x56363cae24d0 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.974+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cb031b0 0x56363cea6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40662/0 (socket says 172.21.15.50:40662) 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 -- 172.21.15.50:0/3316351769 learned_addr learned my addr 172.21.15.50:0/3316351769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cae24d0 0x56363ceadc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 -- 172.21.15.50:0/3316351769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cae24d0 msgr2=0x56363ceadc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cae24d0 0x56363ceadc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 -- 172.21.15.50:0/3316351769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cb031b0 msgr2=0x56363cea6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cb031b0 0x56363cea6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 -- 172.21.15.50:0/3316351769 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363cdd0d50 con 0x56363cada2f0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe5097fa640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 0x7fe4f40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50bfff640 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009da0 tx=0x7fe4fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68cb2ce242dd193 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 0x7fe4f40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.975+0000 7fe5097fa640 1 -- 172.21.15.50:0/2318228772 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe504102fc0 con 0x7fe50c103af0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe4f27fc640 1 -- 172.21.15.50:0/3316351769 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc01e070 con 0x56363cada2f0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe4f27fc640 1 -- 172.21.15.50:0/3316351769 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4fc0198a0 con 0x56363cada2f0 2024-05-17T04:53:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe4f27fc640 1 -- 172.21.15.50:0/3316351769 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0187d0 con 0x56363cada2f0 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe51926c740 1 -- 172.21.15.50:0/3316351769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 msgr2=0x56363cea6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe51926c740 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009da0 tx=0x7fe4fc011810 comp rx=0 tx=0).stop 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.976+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 0x7fe4f40b4d70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56363cea7960 tx=0x7fe500004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 -- 172.21.15.50:0/3316351769 shutdown_connections 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cae24d0 0x56363ceadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cada2f0 0x56363cea6cf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 --2- 172.21.15.50:0/3316351769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cb031b0 0x56363cea6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 -- 172.21.15.50:0/3316351769 >> 172.21.15.50:0/3316351769 conn(0x56363caf7300 msgr2=0x56363cea21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 -- 172.21.15.50:0/3316351769 shutdown_connections 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 -- 172.21.15.50:0/3316351769 wait complete. 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 -- start start 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.977+0000 7fe51926c740 1 Processor -- start 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363ceb0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 0x56363ceb7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56363cd44ab0 con 0x56363cae24d0 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56363cdc54d0 con 0x56363cada2f0 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56363ceae690 con 0x56363cb031b0 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51926c740 10 client.0 _ll_register_callbacks cb 0x56363ce03950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363ceb0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40676/0 (socket says 172.21.15.50:40676) 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51094f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 0x56363ceb7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 -- 172.21.15.50:0/3946276624 learned_addr learned my addr 172.21.15.50:0/3946276624 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 -- 172.21.15.50:0/3946276624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 msgr2=0x56363ceb7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 0x56363ceb7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 -- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 msgr2=0x56363ceb0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363ceb0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe50b7fe640 1 -- 172.21.15.50:0/3946276624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363cdc6340 con 0x56363cae24d0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.978+0000 7fe51094f640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 0x56363ceb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe500023d40 tx=0x7fe500005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe35176aa6585cd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:53:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500033060 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe50001a070 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc0198a0 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe50003a050 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe51926c740 10 client.16629 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56363cb6e790 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.979+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4fc0198a0 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe50001d070 con 0x56363cae24d0 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 10 client.16629 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 20 client.16629 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cac1d80 0x56363cebd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 4 client.16629 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363ce1ac10 con 0x56363cac1d80 2024-05-17T04:53:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.980+0000 7fe50bfff640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cac1d80 0x56363cebd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.981+0000 7fe50bfff640 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cac1d80 0x56363cebd9f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b963a7099557816 server_cookie=ada764955ade5308 in_seq=0 out_seq=0 2024-05-17T04:53:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.981+0000 7fe4eb7fe640 10 client.16629 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/3946276624 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56363ce1ac10 con 0x56363cac1d80 2024-05-17T04:53:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe4eb7fe640 10 client.16629 handle_command_reply: tid=0 2024-05-17T04:53:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 client.16629 shutdown 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 msgr2=0x56363ceb05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe500023d40 tx=0x7fe500005f90 comp rx=0 tx=0).stop 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 shutdown_connections 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cb031b0 0x56363ceb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cac1d80 0x56363cebd9f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb05a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.982+0000 7fe51926c740 1 --2- 172.21.15.50:0/3946276624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363ceb0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.983+0000 7fe4eb7fe640 0 client.16629 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.983+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 >> 172.21.15.50:0/3946276624 conn(0x56363caf7300 msgr2=0x56363cea5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.983+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 shutdown_connections 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.983+0000 7fe51926c740 1 -- 172.21.15.50:0/3946276624 wait complete. 2024-05-17T04:53:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:20.983+0000 7fe51926c740 20 client.16629 trim_cache size 0 max 16384 2024-05-17T04:53:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.006+0000 7fe51926c740 1 Processor -- start 2024-05-17T04:53:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.006+0000 7fe51926c740 1 -- start start 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363cea5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1ac670 con 0x56363cae24d0 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1a9f40 con 0x56363cada2f0 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51926c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1a7d90 con 0x56363cac1d80 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:39156/0 (socket says 172.21.15.50:39156) 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe50bfff640 1 -- 172.21.15.50:0/1795507269 learned_addr learned my addr 172.21.15.50:0/1795507269 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363cea5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.007+0000 7fe51094f640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 -- 172.21.15.50:0/1795507269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 msgr2=0x56363cea5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363cea5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 -- 172.21.15.50:0/1795507269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 msgr2=0x56363ceb7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 -- 172.21.15.50:0/1795507269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363cbe7620 con 0x56363cac1d80 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe51094f640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe50bfff640 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x56363cea7b70 tx=0x7fe4fc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3898dc52952857a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe4f1ffb640 1 -- 172.21.15.50:0/1795507269 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc01e040 con 0x56363cac1d80 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe4f1ffb640 1 -- 172.21.15.50:0/1795507269 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4fc0054d0 con 0x56363cac1d80 2024-05-17T04:53:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.008+0000 7fe4f1ffb640 1 -- 172.21.15.50:0/1795507269 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc02a2b0 con 0x56363cac1d80 2024-05-17T04:53:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 -- 172.21.15.50:0/1795507269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 msgr2=0x56363cea55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x56363cea7b70 tx=0x7fe4fc000bc0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 -- 172.21.15.50:0/1795507269 shutdown_connections 2024-05-17T04:53:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cae24d0 0x56363ceb7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cada2f0 0x56363cea5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.009+0000 7fe51926c740 1 --2- 172.21.15.50:0/1795507269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cac1d80 0x56363cea55e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.010+0000 7fe51926c740 1 -- 172.21.15.50:0/1795507269 >> 172.21.15.50:0/1795507269 conn(0x56363caf7300 msgr2=0x56363cea4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.010+0000 7fe51926c740 1 -- 172.21.15.50:0/1795507269 shutdown_connections 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.010+0000 7fe51926c740 1 -- 172.21.15.50:0/1795507269 wait complete. 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.010+0000 7fe51926c740 1 -- start start 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.010+0000 7fe51926c740 1 Processor -- start 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 0x56363ceb4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 0x56363ceb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cae24d0 0x56363cf23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 0x56363ceb4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1a8400 con 0x56363cac1d80 2024-05-17T04:53:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 0x56363ceb4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40702/0 (socket says 172.21.15.50:40702) 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50bfff640 1 -- 172.21.15.50:0/2067247788 learned_addr learned my addr 172.21.15.50:0/2067247788 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 0x56363ceb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1a9870 con 0x56363cae24d0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1a8ad0 con 0x56363cada2f0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51926c740 10 client.0 _ll_register_callbacks cb 0x7fe500003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe51094f640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cae24d0 0x56363cf23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 -- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cae24d0 msgr2=0x56363cf23d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cae24d0 0x56363cf23d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 -- 172.21.15.50:0/2067247788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 msgr2=0x56363ceb4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 0x56363ceb4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.011+0000 7fe50b7fe640 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363cf24290 con 0x56363cada2f0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe50b7fe640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 0x56363ceb50a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56363cea6060 tx=0x7fe500009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eaaec57945a1d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe4eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500043060 con 0x56363cada2f0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe500029040 con 0x56363cada2f0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc0054d0 con 0x56363cada2f0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe50002e420 con 0x56363cada2f0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe51926c740 10 client.26368 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.012+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56363cf25380 con 0x56363cada2f0 2024-05-17T04:53:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe50003e3f0 con 0x56363cada2f0 2024-05-17T04:53:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe50001d040 con 0x56363cada2f0 2024-05-17T04:53:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 10 client.26368 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 20 client.26368 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cb031b0 0x56363cf27da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 4 client.26368 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:53:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.013+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe4fc0054d0 con 0x56363cb031b0 2024-05-17T04:53:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.014+0000 7fe50bfff640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cb031b0 0x56363cf27da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.014+0000 7fe4eb7fe640 10 client.26368 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.014+0000 7fe50bfff640 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cb031b0 0x56363cf27da0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c67d946ba39d49 server_cookie=e76ca35b7d02ca78 in_seq=0 out_seq=0 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe4eb7fe640 1 -- 172.21.15.50:0/2067247788 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe4fc0054d0 con 0x56363cb031b0 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe4eb7fe640 10 client.26368 handle_command_reply: tid=0 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 client.26368 shutdown 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 msgr2=0x56363ceb50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 0x56363ceb50a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56363cea6060 tx=0x7fe500009f80 comp rx=0 tx=0).stop 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 shutdown_connections 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56363cae24d0 0x56363cf23d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56363cada2f0 0x56363ceb50a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56363cb031b0 0x56363cf27da0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 --2- 172.21.15.50:0/2067247788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56363cac1d80 0x56363ceb4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.030+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 >> 172.21.15.50:0/2067247788 conn(0x56363caf7300 msgr2=0x56363cea9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe4eb7fe640 0 client.26368 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 shutdown_connections 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe51926c740 1 -- 172.21.15.50:0/2067247788 wait complete. 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe51926c740 20 client.26368 trim_cache size 0 max 16384 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:53:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:53:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 msgr2=0x7fe4f40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 0x7fe4f40b4d70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56363cea7960 tx=0x7fe500004730 comp rx=0 tx=0).stop 2024-05-17T04:53:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 msgr2=0x7fe50c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 0x7fe50c1a0ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b30 tx=0x7fe504030be0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 shutdown_connections 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe50c103af0 0x7fe50c1a0ce0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe50c1031a0 0x7fe50c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe4f40b2860 0x7fe4f40b4d70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 --2- 172.21.15.50:0/2318228772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe50c102080 0x7fe50c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.031+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 >> 172.21.15.50:0/2318228772 conn(0x7fe50c0fd880 msgr2=0x7fe50c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.032+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 shutdown_connections 2024-05-17T04:53:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:21.032+0000 7fe5123b2640 1 -- 172.21.15.50:0/2318228772 wait complete. 2024-05-17T04:53:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:22 smithi105 ceph-mon[34795]: pgmap v950: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-17T04:53:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:22 smithi110 ceph-mon[35497]: pgmap v950: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-17T04:53:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:22 smithi050 ceph-mon[31739]: pgmap v950: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.3 MiB/s wr, 315 op/s 2024-05-17T04:53:22.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 865.0 s, 102.8 tps, lat 965.808 ms stddev 298.453 2024-05-17T04:53:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:24 smithi105 ceph-mon[34795]: pgmap v951: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:53:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:24 smithi110 ceph-mon[35497]: pgmap v951: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:53:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:24 smithi050 ceph-mon[31739]: pgmap v951: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-05-17T04:53:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:26 smithi105 ceph-mon[34795]: pgmap v952: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-05-17T04:53:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:26 smithi110 ceph-mon[35497]: pgmap v952: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-05-17T04:53:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:26 smithi050 ceph-mon[31739]: pgmap v952: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.9 MiB/s wr, 301 op/s 2024-05-17T04:53:27.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 870.0 s, 96.6 tps, lat 1058.286 ms stddev 275.911 2024-05-17T04:53:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:28 smithi105 ceph-mon[34795]: pgmap v953: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-17T04:53:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:28 smithi110 ceph-mon[35497]: pgmap v953: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-17T04:53:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:28 smithi050 ceph-mon[31739]: pgmap v953: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.2 MiB/s wr, 332 op/s 2024-05-17T04:53:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:30 smithi105 ceph-mon[34795]: pgmap v954: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-05-17T04:53:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:30 smithi110 ceph-mon[35497]: pgmap v954: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-05-17T04:53:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:30 smithi050 ceph-mon[31739]: pgmap v954: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-05-17T04:53:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:32 smithi105 ceph-mon[34795]: pgmap v955: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:53:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:32 smithi110 ceph-mon[35497]: pgmap v955: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:53:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:32 smithi050 ceph-mon[31739]: pgmap v955: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 2.1 MiB/s wr, 322 op/s 2024-05-17T04:53:32.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 875.0 s, 97.0 tps, lat 1042.494 ms stddev 241.870 2024-05-17T04:53:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 -- 172.21.15.50:0/596868165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 msgr2=0x7f4084101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 --2- 172.21.15.50:0/596868165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f4084101c20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f407400aa10 tx=0x7f407402b720 comp rx=0 tx=0).stop 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 -- 172.21.15.50:0/596868165 shutdown_connections 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 --2- 172.21.15.50:0/596868165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084103280 0x7f4084109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 --2- 172.21.15.50:0/596868165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084102930 0x7f4084102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 --2- 172.21.15.50:0/596868165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f4084101c20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 -- 172.21.15.50:0/596868165 >> 172.21.15.50:0/596868165 conn(0x7f40840fd010 msgr2=0x7f40840ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 -- 172.21.15.50:0/596868165 shutdown_connections 2024-05-17T04:53:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.159+0000 7f4089c12640 1 -- 172.21.15.50:0/596868165 wait complete. 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.160+0000 7f4089c12640 1 Processor -- start 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 -- start start 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f40841a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084103280 0x7f40841ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f408406cde0 con 0x7f4084101810 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f408406cc60 con 0x7f4084103280 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4089c12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f408406cf60 con 0x7f4084102930 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33418/0 (socket says 172.21.15.50:33418) 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f40837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f40841a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4082ffd640 1 -- 172.21.15.50:0/1949012080 learned_addr learned my addr 172.21.15.50:0/1949012080 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.161+0000 7f4083fff640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084103280 0x7f40841ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 -- 172.21.15.50:0/1949012080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084103280 msgr2=0x7f40841ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084103280 0x7f40841ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 -- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 msgr2=0x7f40841a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f40841a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 -- 172.21.15.50:0/1949012080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40841abd40 con 0x7f4084102930 2024-05-17T04:53:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4082ffd640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9f0 tx=0x7f407800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4080ff9640 1 -- 172.21.15.50:0/1949012080 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407800cd80 con 0x7f4084102930 2024-05-17T04:53:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40841abfd0 con 0x7f4084102930 2024-05-17T04:53:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40841ac510 con 0x7f4084102930 2024-05-17T04:53:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.162+0000 7f4080ff9640 1 -- 172.21.15.50:0/1949012080 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40780108c0 con 0x7f4084102930 2024-05-17T04:53:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.163+0000 7f4080ff9640 1 -- 172.21.15.50:0/1949012080 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780187c0 con 0x7f4084102930 2024-05-17T04:53:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4090baf740 1 Processor -- start 2024-05-17T04:53:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4080ff9640 1 -- 172.21.15.50:0/1949012080 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4078018a40 con 0x7f4084102930 2024-05-17T04:53:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4090baf740 1 -- start start 2024-05-17T04:53:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4090baf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4083fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4083fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51526/0 (socket says 172.21.15.50:51526) 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.166+0000 7f4090baf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 -- 172.21.15.50:0/209872620 learned_addr learned my addr 172.21.15.50:0/209872620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4090baf740 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c6a1525cc0 con 0x55c6a12101b0 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c6a1515860 con 0x55c6a11e72f0 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6a15198a0 con 0x55c6a11ef4d0 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4082ffd640 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f40837fe640 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 -- 172.21.15.50:0/209872620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 msgr2=0x55c6a15bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 -- 172.21.15.50:0/209872620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 msgr2=0x55c6a15b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.167+0000 7f4083fff640 1 -- 172.21.15.50:0/209872620 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a14ddd50 con 0x55c6a12101b0 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4083fff640 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f407c009fd0 tx=0x7f407c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349600517586854c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4069ffb640 1 -- 172.21.15.50:0/209872620 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c004d20 con 0x55c6a12101b0 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4080ff9640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 0x7f406c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4069ffb640 1 -- 172.21.15.50:0/209872620 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f407c004050 con 0x55c6a12101b0 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4069ffb640 1 -- 172.21.15.50:0/209872620 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c005260 con 0x55c6a12101b0 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f4080ff9640 1 -- 172.21.15.50:0/1949012080 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f40780e2870 con 0x7f4084102930 2024-05-17T04:53:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.168+0000 7f40837fe640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 0x7f406c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 msgr2=0x55c6a15b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f407c009fd0 tx=0x7f407c0118d0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f40837fe640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 0x7f406c0b4d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4074005ab0 tx=0x7f4074000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 shutdown_connections 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 --2- 172.21.15.50:0/209872620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a12101b0 0x55c6a15b3770 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 >> 172.21.15.50:0/209872620 conn(0x55c6a1204300 msgr2=0x55c6a15af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.169+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 shutdown_connections 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.170+0000 7f4090baf740 1 -- 172.21.15.50:0/209872620 wait complete. 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.170+0000 7f4090baf740 1 -- start start 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.170+0000 7f4090baf740 1 Processor -- start 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c6a1451ab0 con 0x55c6a11e72f0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f40837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33442/0 (socket says 172.21.15.50:33442) 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4082ffd640 1 -- 172.21.15.50:0/737354050 learned_addr learned my addr 172.21.15.50:0/737354050 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c6a14d24d0 con 0x55c6a12101b0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6a15bb690 con 0x55c6a11ef4d0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.171+0000 7f4083fff640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 -- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 msgr2=0x55c6a15c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 -- 172.21.15.50:0/737354050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 msgr2=0x55c6a15bd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15bd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a14d3340 con 0x55c6a11ef4d0 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f40837fe640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4090baf740 10 client.0 _ll_register_callbacks cb 0x55c6a1510950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f407c004050 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4082ffd640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9c0 tx=0x7f40780a9600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbcec8079a25bebe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4062ffd640 10 client.26380 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4062ffd640 1 -- 172.21.15.50:0/737354050 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780e5e60 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4062ffd640 1 -- 172.21.15.50:0/737354050 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4078010640 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.172+0000 7f4062ffd640 1 -- 172.21.15.50:0/737354050 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f40780a9bb0 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 10 client.26380 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55c6a127b790 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4062ffd640 1 -- 172.21.15.50:0/737354050 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f40780a8e10 con 0x55c6a11ef4d0 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 10 client.26380 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 20 client.26380 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11ced80 0x55c6a15ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 4 client.26380 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:53:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a1527c10 con 0x55c6a11ced80 2024-05-17T04:53:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.173+0000 7f40837fe640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11ced80 0x55c6a15ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.174+0000 7f40837fe640 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11ced80 0x55c6a15ca7b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec1e8ee7e42c503 server_cookie=c811cbfb708a06e9 in_seq=0 out_seq=0 2024-05-17T04:53:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.174+0000 7f4062ffd640 10 client.26380 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4062ffd640 1 -- 172.21.15.50:0/737354050 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6a1527c10 con 0x55c6a11ced80 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4062ffd640 10 client.26380 handle_command_reply: tid=0 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 client.26380 shutdown 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 msgr2=0x55c6a15bd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9c0 tx=0x7f40780a9600 comp rx=0 tx=0).stop 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 shutdown_connections 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15bd610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11ced80 0x55c6a15ca7b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 --2- 172.21.15.50:0/737354050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 >> 172.21.15.50:0/737354050 conn(0x55c6a1204300 msgr2=0x55c6a15ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4062ffd640 0 client.26380 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 shutdown_connections 2024-05-17T04:53:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 1 -- 172.21.15.50:0/737354050 wait complete. 2024-05-17T04:53:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.175+0000 7f4090baf740 20 client.26380 trim_cache size 0 max 16384 2024-05-17T04:53:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 Processor -- start 2024-05-17T04:53:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 -- start start 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40700123e0 con 0x55c6a11e72f0 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f407000fcb0 con 0x55c6a12101b0 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.195+0000 7f4090baf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f407000db00 con 0x55c6a11ef4d0 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51550/0 (socket says 172.21.15.50:51550) 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 -- 172.21.15.50:0/1501506505 learned_addr learned my addr 172.21.15.50:0/1501506505 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f4083fff640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 -- 172.21.15.50:0/1501506505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 msgr2=0x55c6a15c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 -- 172.21.15.50:0/1501506505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 msgr2=0x55c6a15b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 -- 172.21.15.50:0/1501506505 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a15bcc40 con 0x55c6a11e72f0 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f4082ffd640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f4083fff640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.196+0000 7f40837fe640 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4084102830 tx=0x7f407403f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd7d6aa693f04ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.197+0000 7f40697fa640 1 -- 172.21.15.50:0/1501506505 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074056070 con 0x55c6a11e72f0 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.197+0000 7f40697fa640 1 -- 172.21.15.50:0/1501506505 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4074004410 con 0x55c6a11e72f0 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.197+0000 7f40697fa640 1 -- 172.21.15.50:0/1501506505 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074036040 con 0x55c6a11e72f0 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- 172.21.15.50:0/1501506505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 msgr2=0x55c6a15af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4084102830 tx=0x7f407403f070 comp rx=0 tx=0).stop 2024-05-17T04:53:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- 172.21.15.50:0/1501506505 shutdown_connections 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a12101b0 0x55c6a15b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 --2- 172.21.15.50:0/1501506505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11e72f0 0x55c6a15af200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- 172.21.15.50:0/1501506505 >> 172.21.15.50:0/1501506505 conn(0x55c6a1204300 msgr2=0x55c6a15aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- 172.21.15.50:0/1501506505 shutdown_connections 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- 172.21.15.50:0/1501506505 wait complete. 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.198+0000 7f4090baf740 1 -- start start 2024-05-17T04:53:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 Processor -- start 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 0x55c6a15c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f407000e170 con 0x55c6a11ef4d0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f407000f5e0 con 0x55c6a11e72f0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f407000e840 con 0x55c6a12101b0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4090baf740 10 client.0 _ll_register_callbacks cb 0x7f407800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4083fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 0x55c6a15c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f40837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.199+0000 7f4082ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:49808/0 (socket says 172.21.15.50:49808) 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4083fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 0x55c6a15c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:33462/0 (socket says 172.21.15.50:33462) 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 -- 172.21.15.50:0/2098634037 learned_addr learned my addr 172.21.15.50:0/2098634037 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 -- 172.21.15.50:0/2098634037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 msgr2=0x55c6a15c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 0x55c6a15c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 -- 172.21.15.50:0/2098634037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 msgr2=0x55c6a15c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 -- 172.21.15.50:0/2098634037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a12f4620 con 0x55c6a11e72f0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4082ffd640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f40837fe640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f407402bc40 tx=0x7f4074030800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61521442a2198ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407405e060 con 0x55c6a11e72f0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f40740512d0 con 0x55c6a11e72f0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.200+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4074004410 con 0x55c6a11e72f0 2024-05-17T04:53:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4074036070 con 0x55c6a11e72f0 2024-05-17T04:53:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 10 client.26405 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c6a15c4670 con 0x55c6a11e72f0 2024-05-17T04:53:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4074004410 con 0x55c6a11e72f0 2024-05-17T04:53:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f4074059070 con 0x55c6a11e72f0 2024-05-17T04:53:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 10 client.26405 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 10 client.26405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 20 client.26405 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11a41c0 0x55c6a1634f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 4 client.26405 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.201+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6a15213f0 con 0x55c6a11a41c0 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.202+0000 7f4082ffd640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11a41c0 0x55c6a1634f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.202+0000 7f4062ffd640 10 client.26405 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.202+0000 7f4082ffd640 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11a41c0 0x55c6a1634f60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b6abfacacfe2fa server_cookie=27ae33099c79493f in_seq=0 out_seq=0 2024-05-17T04:53:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4062ffd640 1 -- 172.21.15.50:0/2098634037 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6a15213f0 con 0x55c6a11a41c0 2024-05-17T04:53:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4062ffd640 10 client.26405 handle_command_reply: tid=0 2024-05-17T04:53:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 client.26405 shutdown 2024-05-17T04:53:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 msgr2=0x55c6a15c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f407402bc40 tx=0x7f4074030800 comp rx=0 tx=0).stop 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 shutdown_connections 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a12101b0 0x55c6a15c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55c6a11a41c0 0x55c6a1634f60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c6a11ef4d0 0x55c6a15c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.222+0000 7f4090baf740 1 --2- 172.21.15.50:0/2098634037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c6a11e72f0 0x55c6a15c1b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.223+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 >> 172.21.15.50:0/2098634037 conn(0x55c6a1204300 msgr2=0x55c6a15ad7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.223+0000 7f4062ffd640 0 client.26405 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.223+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 shutdown_connections 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.223+0000 7f4090baf740 1 -- 172.21.15.50:0/2098634037 wait complete. 2024-05-17T04:53:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.223+0000 7f4090baf740 20 client.26405 trim_cache size 0 max 16384 2024-05-17T04:53:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:53:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:53:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 msgr2=0x7f406c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 0x7f406c0b4d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4074005ab0 tx=0x7f4074000bc0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 msgr2=0x7f40841a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f407800e9f0 tx=0x7f407800eeb0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 shutdown_connections 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4084103280 0x7f40841ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4084102930 0x7f40841a4740 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f406c0b2860 0x7f406c0b4d70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 --2- 172.21.15.50:0/1949012080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4084101810 0x7f40841a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 >> 172.21.15.50:0/1949012080 conn(0x7f40840fd010 msgr2=0x7f4084073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 shutdown_connections 2024-05-17T04:53:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:34.224+0000 7f4089c12640 1 -- 172.21.15.50:0/1949012080 wait complete. 2024-05-17T04:53:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:34 smithi105 ceph-mon[34795]: pgmap v956: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.4 MiB/s wr, 363 op/s 2024-05-17T04:53:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:34 smithi110 ceph-mon[35497]: pgmap v956: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.4 MiB/s wr, 363 op/s 2024-05-17T04:53:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:34 smithi050 ceph-mon[31739]: pgmap v956: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 2.4 MiB/s wr, 363 op/s 2024-05-17T04:53:36.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:36 smithi105 ceph-mon[34795]: pgmap v957: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-05-17T04:53:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:36 smithi110 ceph-mon[35497]: pgmap v957: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-05-17T04:53:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:36 smithi050 ceph-mon[31739]: pgmap v957: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.3 MiB/s wr, 326 op/s 2024-05-17T04:53:37.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 880.0 s, 100.2 tps, lat 964.118 ms stddev 236.540 2024-05-17T04:53:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:38 smithi105 ceph-mon[34795]: pgmap v958: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-05-17T04:53:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:38 smithi110 ceph-mon[35497]: pgmap v958: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-05-17T04:53:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:38 smithi050 ceph-mon[31739]: pgmap v958: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.8 MiB/s wr, 310 op/s 2024-05-17T04:53:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:40 smithi105 ceph-mon[34795]: pgmap v959: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-17T04:53:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:40 smithi110 ceph-mon[35497]: pgmap v959: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-17T04:53:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:40 smithi050 ceph-mon[31739]: pgmap v959: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-17T04:53:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:42 smithi105 ceph-mon[34795]: pgmap v960: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.1 MiB/s wr, 310 op/s 2024-05-17T04:53:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:42 smithi110 ceph-mon[35497]: pgmap v960: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.1 MiB/s wr, 310 op/s 2024-05-17T04:53:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:42 smithi050 ceph-mon[31739]: pgmap v960: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.1 MiB/s wr, 310 op/s 2024-05-17T04:53:42.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 885.0 s, 98.8 tps, lat 1019.347 ms stddev 234.669 2024-05-17T04:53:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:44 smithi105 ceph-mon[34795]: pgmap v961: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 333 op/s 2024-05-17T04:53:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:44 smithi110 ceph-mon[35497]: pgmap v961: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 333 op/s 2024-05-17T04:53:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:44 smithi050 ceph-mon[31739]: pgmap v961: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.1 MiB/s wr, 333 op/s 2024-05-17T04:53:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:46 smithi105 ceph-mon[34795]: pgmap v962: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.0 MiB/s wr, 298 op/s 2024-05-17T04:53:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:46 smithi110 ceph-mon[35497]: pgmap v962: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.0 MiB/s wr, 298 op/s 2024-05-17T04:53:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:46 smithi050 ceph-mon[31739]: pgmap v962: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 2.0 MiB/s wr, 298 op/s 2024-05-17T04:53:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 -- 172.21.15.50:0/393471400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50102080 msgr2=0x7f4f50102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 --2- 172.21.15.50:0/393471400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50102080 0x7f4f50102490 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009900 tx=0x7f4f4402a410 comp rx=0 tx=0).stop 2024-05-17T04:53:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 -- 172.21.15.50:0/393471400 shutdown_connections 2024-05-17T04:53:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 --2- 172.21.15.50:0/393471400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50103af0 0x7f4f5010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 --2- 172.21.15.50:0/393471400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 0x7f4f501035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 --2- 172.21.15.50:0/393471400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50102080 0x7f4f50102490 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 -- 172.21.15.50:0/393471400 >> 172.21.15.50:0/393471400 conn(0x7f4f500fd880 msgr2=0x7f4f500ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.363+0000 7f4f571f5640 1 -- 172.21.15.50:0/393471400 shutdown_connections 2024-05-17T04:53:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.364+0000 7f4f571f5640 1 -- 172.21.15.50:0/393471400 wait complete. 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.364+0000 7f4f571f5640 1 Processor -- start 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.364+0000 7f4f571f5640 1 -- start start 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 0x7f4f50199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50103af0 0x7f4f501a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5010da80 con 0x7f4f50103af0 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5010d900 con 0x7f4f50102080 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f571f5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5010dc00 con 0x7f4f501031a0 2024-05-17T04:53:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f55792640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50103af0 0x7f4f501a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39848/0 (socket says 172.21.15.50:39848) 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 -- 172.21.15.50:0/940380654 learned_addr learned my addr 172.21.15.50:0/940380654 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 0x7f4f50199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 -- 172.21.15.50:0/940380654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 msgr2=0x7f4f50199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 0x7f4f50199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 -- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50103af0 msgr2=0x7f4f501a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50103af0 0x7f4f501a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.365+0000 7f4f54f91640 1 -- 172.21.15.50:0/940380654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f501a1390 con 0x7f4f50102080 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f54f91640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005760 tx=0x7f4f44003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f4dffb640 1 -- 172.21.15.50:0/940380654 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4403f070 con 0x7f4f50102080 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f4dffb640 1 -- 172.21.15.50:0/940380654 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f44031ba0 con 0x7f4f50102080 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f4dffb640 1 -- 172.21.15.50:0/940380654 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44035030 con 0x7f4f50102080 2024-05-17T04:53:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f501a1620 con 0x7f4f50102080 2024-05-17T04:53:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.366+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f501a1ac0 con 0x7f4f50102080 2024-05-17T04:53:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 Processor -- start 2024-05-17T04:53:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 -- start start 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 0x56052e7ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 0x56052e7d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56052e73bcc0 con 0x56052e4054d0 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56052e72b860 con 0x56052e3fd2f0 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f5e194740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56052e72f8a0 con 0x56052e4261b0 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.369+0000 7f4f4dffb640 1 -- 172.21.15.50:0/940380654 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4f44042070 con 0x7f4f50102080 2024-05-17T04:53:47.370 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f55792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f55792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:46590/0 (socket says 172.21.15.50:46590) 2024-05-17T04:53:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f55792640 1 -- 172.21.15.50:0/4166312392 learned_addr learned my addr 172.21.15.50:0/4166312392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f54f91640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 0x56052e7ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 0x56052e7d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.370+0000 7f4f55792640 1 -- 172.21.15.50:0/4166312392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 msgr2=0x56052e7ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f55792640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 0x56052e7ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f55792640 1 -- 172.21.15.50:0/4166312392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 msgr2=0x56052e7d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f55792640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 0x56052e7d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f55792640 1 -- 172.21.15.50:0/4166312392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052e6f3d50 con 0x56052e4261b0 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 0x56052e7d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f54f91640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 0x56052e7ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f55792640 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fd0 tx=0x7f4f480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8c18db453a5298 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f4dffb640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 0x7f4f400b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f3affd640 1 -- 172.21.15.50:0/4166312392 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004d20 con 0x56052e4261b0 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f3affd640 1 -- 172.21.15.50:0/4166312392 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f48004050 con 0x56052e4261b0 2024-05-17T04:53:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f3affd640 1 -- 172.21.15.50:0/4166312392 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48005260 con 0x56052e4261b0 2024-05-17T04:53:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f4dffb640 1 -- 172.21.15.50:0/940380654 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f44102d50 con 0x7f4f50102080 2024-05-17T04:53:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.371+0000 7f4f54f91640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 0x7f4f400b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.372+0000 7f4f5e194740 1 -- 172.21.15.50:0/4166312392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 msgr2=0x56052e7cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fd0 tx=0x7f4f480118d0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f54f91640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 0x7f4f400b4cf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56052e7cef80 tx=0x7f4f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 -- 172.21.15.50:0/4166312392 shutdown_connections 2024-05-17T04:53:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4054d0 0x56052e7d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3fd2f0 0x56052e7ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 --2- 172.21.15.50:0/4166312392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4261b0 0x56052e7cdb80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 -- 172.21.15.50:0/4166312392 >> 172.21.15.50:0/4166312392 conn(0x56052e41a300 msgr2=0x56052e7c51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 -- 172.21.15.50:0/4166312392 shutdown_connections 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 -- 172.21.15.50:0/4166312392 wait complete. 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 -- start start 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.373+0000 7f4f5e194740 1 Processor -- start 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 0x56052e7d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f55792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 0x56052e7d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f55792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 0x56052e7d7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:46596/0 (socket says 172.21.15.50:46596) 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f55792640 1 -- 172.21.15.50:0/2122228688 learned_addr learned my addr 172.21.15.50:0/2122228688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4261b0 0x56052e7de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56052e667ab0 con 0x56052e4261b0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56052e6e84d0 con 0x56052e4054d0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56052e7d5aa0 con 0x56052e3fd2f0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f55792640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4261b0 0x56052e7de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.374+0000 7f4f5e194740 10 client.0 _ll_register_callbacks cb 0x56052e726950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 -- 172.21.15.50:0/2122228688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 msgr2=0x56052e7d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 0x56052e7d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 -- 172.21.15.50:0/2122228688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4261b0 msgr2=0x56052e7de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4261b0 0x56052e7de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052e6e9340 con 0x56052e4054d0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f48004050 con 0x56052e4054d0 2024-05-17T04:53:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d79b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f5019a8f0 tx=0x7f4f3c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd48d6739ce4ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f2ffff640 10 client.26417 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f2ffff640 1 -- 172.21.15.50:0/2122228688 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00fa60 con 0x56052e4054d0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f2ffff640 1 -- 172.21.15.50:0/2122228688 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f3c00fc00 con 0x56052e4054d0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f2ffff640 1 -- 172.21.15.50:0/2122228688 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c00fe60 con 0x56052e4054d0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f5e194740 10 client.26417 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.375+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=43}) -- 0x56052e491790 con 0x56052e4054d0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f2ffff640 1 -- 172.21.15.50:0/2122228688 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f4f3c004030 con 0x56052e4054d0 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 10 client.26417 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 20 client.26417 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e3e4d80 0x56052e7e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 4 client.26417 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052e73dc10 con 0x56052e3e4d80 2024-05-17T04:53:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f55792640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e3e4d80 0x56052e7e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f55792640 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e3e4d80 0x56052e7e4af0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f6240f239f1ea server_cookie=f77168c98f6b36b2 in_seq=0 out_seq=0 2024-05-17T04:53:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.376+0000 7f4f2ffff640 10 client.26417 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f2ffff640 1 -- 172.21.15.50:0/2122228688 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56052e73dc10 con 0x56052e3e4d80 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f2ffff640 10 client.26417 handle_command_reply: tid=0 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 client.26417 shutdown 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 msgr2=0x56052e7d79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d79b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f5019a8f0 tx=0x7f4f3c012b60 comp rx=0 tx=0).stop 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 shutdown_connections 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e3e4d80 0x56052e7e4af0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f2ffff640 0 client.26417 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e4261b0 0x56052e7de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d79b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 --2- 172.21.15.50:0/2122228688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3fd2f0 0x56052e7d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.377+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 >> 172.21.15.50:0/2122228688 conn(0x56052e41a300 msgr2=0x56052e7c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.378+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 shutdown_connections 2024-05-17T04:53:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.378+0000 7f4f5e194740 1 -- 172.21.15.50:0/2122228688 wait complete. 2024-05-17T04:53:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.378+0000 7f4f5e194740 20 client.26417 trim_cache size 0 max 16384 2024-05-17T04:53:47.397 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.397+0000 7f4f5e194740 1 Processor -- start 2024-05-17T04:53:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 -- start start 2024-05-17T04:53:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 0x56052e7c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501ac740 con 0x56052e3fd2f0 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501aa010 con 0x56052e4054d0 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f5e194740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501a7e60 con 0x56052e3e4d80 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f55792640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 0x56052e7c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43814/0 (socket says 172.21.15.50:43814) 2024-05-17T04:53:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.400+0000 7f4f4ffff640 1 -- 172.21.15.50:0/121487462 learned_addr learned my addr 172.21.15.50:0/121487462 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 -- 172.21.15.50:0/121487462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 msgr2=0x56052e7d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 -- 172.21.15.50:0/121487462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 msgr2=0x56052e7de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 -- 172.21.15.50:0/121487462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052e50a620 con 0x56052e3e4d80 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f55792640 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 0x56052e7c5160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fa0 tx=0x7f4f480096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09b3e259db07d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f3a7fc640 1 -- 172.21.15.50:0/121487462 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4802c070 con 0x56052e3e4d80 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f3a7fc640 1 -- 172.21.15.50:0/121487462 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f48004cc0 con 0x56052e3e4d80 2024-05-17T04:53:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.401+0000 7f4f3a7fc640 1 -- 172.21.15.50:0/121487462 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4802e700 con 0x56052e3e4d80 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- 172.21.15.50:0/121487462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 msgr2=0x56052e7c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 0x56052e7c5160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fa0 tx=0x7f4f480096c0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- 172.21.15.50:0/121487462 shutdown_connections 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e4054d0 0x56052e7d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 --2- 172.21.15.50:0/121487462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e3e4d80 0x56052e7c5160 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- 172.21.15.50:0/121487462 >> 172.21.15.50:0/121487462 conn(0x56052e41a300 msgr2=0x56052e7c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- 172.21.15.50:0/121487462 shutdown_connections 2024-05-17T04:53:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- 172.21.15.50:0/121487462 wait complete. 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 -- start start 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.403+0000 7f4f5e194740 1 Processor -- start 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4054d0 0x56052e7dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501a84d0 con 0x56052e3fd2f0 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501a9940 con 0x56052e3e4d80 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501a8ba0 con 0x56052e4054d0 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f5e194740 10 client.0 _ll_register_callbacks cb 0x7f4f3c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:53:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:39878/0 (socket says 172.21.15.50:39878) 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 -- 172.21.15.50:0/619962099 learned_addr learned my addr 172.21.15.50:0/619962099 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4054d0 0x56052e7dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 -- 172.21.15.50:0/619962099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4054d0 msgr2=0x56052e7dd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4054d0 0x56052e7dd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 -- 172.21.15.50:0/619962099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 msgr2=0x56052e7da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.404+0000 7f4f55792640 1 -- 172.21.15.50:0/619962099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052e7ddc90 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f55792640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009730 tx=0x7f4f48005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1a934fe312e168 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4802c070 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4f48003cf0 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f48004cc0 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f4802e480 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f5e194740 10 client.26423 fetch_fsmap learned FSMap version 43 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56052e84cfc0 con 0x56052e3e4d80 2024-05-17T04:53:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.405+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f4802eab0 con 0x56052e3e4d80 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f4f4800f8f0 con 0x56052e3e4d80 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 10 client.26423 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 10 client.26423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 20 client.26423 populate_metadata read hostname 'smithi050' 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e4261b0 0x56052e84f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 4 client.26423 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4f48004cc0 con 0x56052e4261b0 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e4261b0 0x56052e84f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f2ffff640 10 client.26423 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.406+0000 7f4f4ffff640 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e4261b0 0x56052e84f740 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2bbc9a7027c27ec server_cookie=787e0c57d4c25d47 in_seq=0 out_seq=0 2024-05-17T04:53:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.427+0000 7f4f2ffff640 1 -- 172.21.15.50:0/619962099 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f4f48004cc0 con 0x56052e4261b0 2024-05-17T04:53:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.427+0000 7f4f2ffff640 10 client.26423 handle_command_reply: tid=0 2024-05-17T04:53:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.427+0000 7f4f5e194740 1 client.26423 shutdown 2024-05-17T04:53:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.427+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 msgr2=0x56052e7da220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.427+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009730 tx=0x7f4f48005fd0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 shutdown_connections 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56052e4054d0 0x56052e7dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x56052e4261b0 0x56052e84f740 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56052e3fd2f0 0x56052e7da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 --2- 172.21.15.50:0/619962099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56052e3e4d80 0x56052e7da220 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f2ffff640 0 client.26423 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 >> 172.21.15.50:0/619962099 conn(0x56052e41a300 msgr2=0x56052e7c37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 shutdown_connections 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 1 -- 172.21.15.50:0/619962099 wait complete. 2024-05-17T04:53:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.428+0000 7f4f5e194740 20 client.26423 trim_cache size 0 max 16384 2024-05-17T04:53:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:53:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:53:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 msgr2=0x7f4f400b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 0x7f4f400b4cf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56052e7cef80 tx=0x7f4f44009260 comp rx=0 tx=0).stop 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 msgr2=0x7f4f50199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005760 tx=0x7f4f44003c10 comp rx=0 tx=0).stop 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 shutdown_connections 2024-05-17T04:53:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f4f400b27e0 0x7f4f400b4cf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.429+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f50103af0 0x7f4f501a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.430+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f501031a0 0x7f4f50199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.430+0000 7f4f571f5640 1 --2- 172.21.15.50:0/940380654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f50102080 0x7f4f50199850 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.430+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 >> 172.21.15.50:0/940380654 conn(0x7f4f500fd880 msgr2=0x7f4f500ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.430+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 shutdown_connections 2024-05-17T04:53:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:53:47.430+0000 7f4f571f5640 1 -- 172.21.15.50:0/940380654 wait complete. 2024-05-17T04:53:47.798 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 890.0 s, 95.6 tps, lat 1055.258 ms stddev 253.346 2024-05-17T04:53:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:48 smithi105 ceph-mon[34795]: pgmap v963: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-17T04:53:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:48 smithi110 ceph-mon[35497]: pgmap v963: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-17T04:53:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:48 smithi050 ceph-mon[31739]: pgmap v963: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-17T04:53:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:50 smithi105 ceph-mon[34795]: pgmap v964: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-05-17T04:53:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:50 smithi110 ceph-mon[35497]: pgmap v964: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-05-17T04:53:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:50 smithi050 ceph-mon[31739]: pgmap v964: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-05-17T04:53:51.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:51 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:53:51.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:51 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:53:51.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:51 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:53:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:52 smithi105 ceph-mon[34795]: pgmap v965: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:53:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:53:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:53:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:53:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:52 smithi110 ceph-mon[35497]: pgmap v965: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:53:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:53:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:53:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:53:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:52 smithi050 ceph-mon[31739]: pgmap v965: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:53:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:53:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:53:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:53:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 895.0 s, 108.2 tps, lat 920.899 ms stddev 312.559 2024-05-17T04:53:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:54 smithi105 ceph-mon[34795]: pgmap v966: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 343 op/s 2024-05-17T04:53:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:54 smithi110 ceph-mon[35497]: pgmap v966: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 343 op/s 2024-05-17T04:53:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:54 smithi050 ceph-mon[31739]: pgmap v966: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 2.3 MiB/s wr, 343 op/s 2024-05-17T04:53:55.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:55 smithi105 ceph-mon[34795]: pgmap v967: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-05-17T04:53:55.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:55 smithi110 ceph-mon[35497]: pgmap v967: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-05-17T04:53:55.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:55 smithi050 ceph-mon[31739]: pgmap v967: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.9 MiB/s wr, 292 op/s 2024-05-17T04:53:57.797 INFO:teuthology.orchestra.run.smithi050.stderr:progress: 900.0 s, 95.6 tps, lat 1060.210 ms stddev 260.044 2024-05-17T04:53:58.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:53:58 smithi105 ceph-mon[34795]: pgmap v968: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-05-17T04:53:58.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:53:58 smithi110 ceph-mon[35497]: pgmap v968: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-05-17T04:53:58.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:53:58 smithi050 ceph-mon[31739]: pgmap v968: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.8 MiB/s wr, 299 op/s 2024-05-17T04:53:58.780 INFO:teuthology.orchestra.run.smithi050.stdout:transaction type: 2024-05-17T04:53:58.780 INFO:teuthology.orchestra.run.smithi050.stdout:scaling factor: 32 2024-05-17T04:53:58.780 INFO:teuthology.orchestra.run.smithi050.stdout:query mode: simple 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:number of clients: 100 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:number of threads: 4 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:duration: 900 s 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:number of transactions actually processed: 114187 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:latency average = 788.404 ms 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:latency stddev = 2773.398 ms 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:tps = 126.735995 (including connections establishing) 2024-05-17T04:53:58.781 INFO:teuthology.orchestra.run.smithi050.stdout:tps = 126.737606 (excluding connections establishing) 2024-05-17T04:53:58.822 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 -- 172.21.15.50:0/2366969764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968102600 msgr2=0x7f0968102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 --2- 172.21.15.50:0/2366969764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968102600 0x7f0968102a10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0958009900 tx=0x7f095802a410 comp rx=0 tx=0).stop 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 -- 172.21.15.50:0/2366969764 shutdown_connections 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 --2- 172.21.15.50:0/2366969764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968103b10 0x7f096810a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 --2- 172.21.15.50:0/2366969764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102fe0 0x7f0968103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 --2- 172.21.15.50:0/2366969764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968102600 0x7f0968102a10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 -- 172.21.15.50:0/2366969764 >> 172.21.15.50:0/2366969764 conn(0x7f09680fd010 msgr2=0x7f09680ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 -- 172.21.15.50:0/2366969764 shutdown_connections 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.555+0000 7f096e111640 1 -- 172.21.15.50:0/2366969764 wait complete. 2024-05-17T04:54:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 Processor -- start 2024-05-17T04:54:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 -- start start 2024-05-17T04:54:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 0x7f0968199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 0x7f09681a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f096810da80 con 0x7f0968103b10 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f096810d900 con 0x7f0968102600 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f096e111640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f096810dc00 con 0x7f0968102fe0 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f095ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f09677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 0x7f0968199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f0967fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 0x7f09681a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f095ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:34374/0 (socket says 172.21.15.50:34374) 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f09677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 0x7f0968199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:43912/0 (socket says 172.21.15.50:43912) 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.556+0000 7f095ffff640 1 -- 172.21.15.50:0/3381529220 learned_addr learned my addr 172.21.15.50:0/3381529220 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 -- 172.21.15.50:0/3381529220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 msgr2=0x7f0968199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 0x7f0968199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 -- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 msgr2=0x7f09681a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 0x7f09681a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 -- 172.21.15.50:0/3381529220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681a1380 con 0x7f0968102fe0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f0967fff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 0x7f09681a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f095ffff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f095000e9c0 tx=0x7f095000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f09657fa640 1 -- 172.21.15.50:0/3381529220 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095000cd80 con 0x7f0968102fe0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681a1670 con 0x7f0968102fe0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f09657fa640 1 -- 172.21.15.50:0/3381529220 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09500054b0 con 0x7f0968102fe0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f09657fa640 1 -- 172.21.15.50:0/3381529220 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950010430 con 0x7f0968102fe0 2024-05-17T04:54:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.557+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a1c10 con 0x7f0968102fe0 2024-05-17T04:54:00.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:00 smithi105 ceph-mon[34795]: pgmap v969: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 Processor -- start 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 -- start start 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f09657fa640 1 -- 172.21.15.50:0/3381529220 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f09500105d0 con 0x7f0968102fe0 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dbde426cc0 con 0x55dbde0f04d0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dbde416860 con 0x55dbde1111b0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0974fd2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dbde41a8a0 con 0x55dbde0e82f0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f095ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f09677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f0967fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f09677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:34388/0 (socket says 172.21.15.50:34388) 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.560+0000 7f09677fe640 1 -- 172.21.15.50:0/1748731187 learned_addr learned my addr 172.21.15.50:0/1748731187 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 -- 172.21.15.50:0/1748731187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 msgr2=0x55dbde4b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 -- 172.21.15.50:0/1748731187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 msgr2=0x55dbde4bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 -- 172.21.15.50:0/1748731187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbde3ded50 con 0x55dbde0e82f0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09657fa640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 0x7f09540b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09677fe640 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f096806b0e0 tx=0x7f09580314f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a575824ef98a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f0967fff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 0x7f09540b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f09657fa640 1 -- 172.21.15.50:0/3381529220 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f09500e2bc0 con 0x7f0968102fe0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f095dffb640 1 -- 172.21.15.50:0/1748731187 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958040070 con 0x55dbde0e82f0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f095dffb640 1 -- 172.21.15.50:0/1748731187 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09580044c0 con 0x55dbde0e82f0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.561+0000 7f095dffb640 1 -- 172.21.15.50:0/1748731187 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958035040 con 0x55dbde0e82f0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 -- 172.21.15.50:0/1748731187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 msgr2=0x55dbde4b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0967fff640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 0x7f09540b4cf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55dbde4b5960 tx=0x7f0960009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f096806b0e0 tx=0x7f09580314f0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 -- 172.21.15.50:0/1748731187 shutdown_connections 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0e82f0 0x55dbde4b4cf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1748731187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 -- 172.21.15.50:0/1748731187 >> 172.21.15.50:0/1748731187 conn(0x55dbde105300 msgr2=0x55dbde4b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 -- 172.21.15.50:0/1748731187 shutdown_connections 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.562+0000 7f0974fd2740 1 -- 172.21.15.50:0/1748731187 wait complete. 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 -- start start 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 Processor -- start 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dbde352ab0 con 0x55dbde0e82f0 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dbde3d34d0 con 0x55dbde1111b0 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dbde4bc630 con 0x55dbde0f04d0 2024-05-17T04:54:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f0974fd2740 10 client.0 _ll_register_callbacks cb 0x55dbde411950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.563+0000 7f09677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41030/0 (socket says 172.21.15.50:41030) 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 -- 172.21.15.50:0/1543658351 learned_addr learned my addr 172.21.15.50:0/1543658351 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0967fff640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f095ffff640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 -- 172.21.15.50:0/1543658351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 msgr2=0x55dbde4be540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4be540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 -- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 msgr2=0x55dbde4c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 -- 172.21.15.50:0/1543658351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbde3d4340 con 0x55dbde0e82f0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f09677fe640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0958005f70 tx=0x7f0958031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daad26fdd12813c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095804d060 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0958005440 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbde428c10 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0958040070 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0974fd2740 10 client.16683 fetch_fsmap learned FSMap version 43 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.564+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dbde17c790 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55dbde428c10 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f095803c440 con 0x55dbde0e82f0 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 10 client.16683 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 20 client.16683 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0cfd80 0x55dbde4cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 4 client.16683 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbde4223f0 con 0x55dbde0cfd80 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f095ffff640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0cfd80 0x55dbde4cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.565+0000 7f0946ffd640 10 client.16683 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.566+0000 7f095ffff640 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0cfd80 0x55dbde4cb380 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825d8a9e0cfb3cf0 server_cookie=32e236e4f77bc4b0 in_seq=0 out_seq=0 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.566+0000 7f0946ffd640 1 -- 172.21.15.50:0/1543658351 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbde4223f0 con 0x55dbde0cfd80 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.566+0000 7f0946ffd640 10 client.16683 handle_command_reply: tid=0 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.566+0000 7f0974fd2740 1 client.16683 shutdown 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 msgr2=0x55dbde4be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0958005f70 tx=0x7f0958031c90 comp rx=0 tx=0).stop 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 shutdown_connections 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde1111b0 0x55dbde4c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4be540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0cfd80 0x55dbde4cb380 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 --2- 172.21.15.50:0/1543658351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0e82f0 0x55dbde4be000 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0946ffd640 0 client.16683 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 >> 172.21.15.50:0/1543658351 conn(0x55dbde105300 msgr2=0x55dbde4b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 shutdown_connections 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 1 -- 172.21.15.50:0/1543658351 wait complete. 2024-05-17T04:54:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.567+0000 7f0974fd2740 20 client.16683 trim_cache size 0 max 16384 2024-05-17T04:54:00.574 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:00 smithi110 ceph-mon[35497]: pgmap v969: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:54:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:00 smithi050 ceph-mon[31739]: pgmap v969: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.2 MiB/s wr, 311 op/s 2024-05-17T04:54:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 Processor -- start 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 -- start start 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde1111b0 0x55dbde4cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09681ae860 con 0x55dbde0f04d0 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09681ac130 con 0x55dbde0e82f0 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0974fd2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f09681a9f80 con 0x55dbde1111b0 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41036/0 (socket says 172.21.15.50:41036) 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 -- 172.21.15.50:0/333285282 learned_addr learned my addr 172.21.15.50:0/333285282 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f0967fff640 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde1111b0 0x55dbde4cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f09677fe640 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 -- 172.21.15.50:0/333285282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde1111b0 msgr2=0x55dbde4cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde1111b0 0x55dbde4cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 -- 172.21.15.50:0/333285282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 msgr2=0x55dbde4b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.583+0000 7f095ffff640 1 -- 172.21.15.50:0/333285282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbde1f5620 con 0x55dbde0f04d0 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.584+0000 7f095ffff640 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f09500e0310 tx=0x7f095000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6d863fcf889a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.584+0000 7f095d7fa640 1 -- 172.21.15.50:0/333285282 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500e7040 con 0x55dbde0f04d0 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.584+0000 7f095d7fa640 1 -- 172.21.15.50:0/333285282 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09500a9940 con 0x55dbde0f04d0 2024-05-17T04:54:00.585 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.584+0000 7f095d7fa640 1 -- 172.21.15.50:0/333285282 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500b2c90 con 0x55dbde0f04d0 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- 172.21.15.50:0/333285282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 msgr2=0x55dbde4b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f09500e0310 tx=0x7f095000cf20 comp rx=0 tx=0).stop 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- 172.21.15.50:0/333285282 shutdown_connections 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde1111b0 0x55dbde4cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde0f04d0 0x55dbde4b3860 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 --2- 172.21.15.50:0/333285282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- 172.21.15.50:0/333285282 >> 172.21.15.50:0/333285282 conn(0x55dbde105300 msgr2=0x55dbde4be3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- 172.21.15.50:0/333285282 shutdown_connections 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- 172.21.15.50:0/333285282 wait complete. 2024-05-17T04:54:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.585+0000 7f0974fd2740 1 -- start start 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 Processor -- start 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09681aa5f0 con 0x55dbde1111b0 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09681aba60 con 0x55dbde0e82f0 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f09681aacc0 con 0x55dbde0f04d0 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0974fd2740 10 client.0 _ll_register_callbacks cb 0x7f09500e6510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:41046/0 (socket says 172.21.15.50:41046) 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 -- 172.21.15.50:0/2288365718 learned_addr learned my addr 172.21.15.50:0/2288365718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f095ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f09677fe640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 -- 172.21.15.50:0/2288365718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 msgr2=0x55dbde4c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 -- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 msgr2=0x55dbde4c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 -- 172.21.15.50:0/2288365718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbde531c80 con 0x55dbde1111b0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f095ffff640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0967fff640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f096000fb40 tx=0x7f0960004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afb9dc3ce77a675 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.586+0000 7f0946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096002f060 con 0x55dbde1111b0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0960011950 con 0x55dbde1111b0 2024-05-17T04:54:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09500a9940 con 0x55dbde1111b0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0960036050 con 0x55dbde1111b0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 10 client.16689 fetch_fsmap learned FSMap version 43 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dbde532fc0 con 0x55dbde1111b0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09500a9940 con 0x55dbde1111b0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f096001fb30 con 0x55dbde1111b0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 10 client.16689 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.587+0000 7f0974fd2740 20 client.16689 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0a51c0 0x55dbde535950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f0974fd2740 4 client.16689 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbde4bc060 con 0x55dbde0a51c0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f09677fe640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0a51c0 0x55dbde535950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f0946ffd640 10 client.16689 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.588+0000 7f09677fe640 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0a51c0 0x55dbde535950 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7222ef605e9950b7 server_cookie=c37fbe3d61498da7 in_seq=0 out_seq=0 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0946ffd640 1 -- 172.21.15.50:0/2288365718 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dbde4bc060 con 0x55dbde0a51c0 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0946ffd640 10 client.16689 handle_command_reply: tid=0 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0974fd2740 1 client.16689 shutdown 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 msgr2=0x55dbde531690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f096000fb40 tx=0x7f0960004a40 comp rx=0 tx=0).stop 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 shutdown_connections 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.601+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55dbde0a51c0 0x55dbde535950 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0946ffd640 0 client.16689 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dbde1111b0 0x55dbde531690 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbde0f04d0 0x55dbde4c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 --2- 172.21.15.50:0/2288365718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dbde0e82f0 0x55dbde4c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 >> 172.21.15.50:0/2288365718 conn(0x55dbde105300 msgr2=0x55dbde4be3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 shutdown_connections 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 1 -- 172.21.15.50:0/2288365718 wait complete. 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f0974fd2740 20 client.16689 trim_cache size 0 max 16384 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:54:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 msgr2=0x7f09540b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 0x7f09540b4cf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55dbde4b5960 tx=0x7f0960009500 comp rx=0 tx=0).stop 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 msgr2=0x7f0968199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f095000e9c0 tx=0x7f095000ee80 comp rx=0 tx=0).stop 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 shutdown_connections 2024-05-17T04:54:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f09540b27e0 0x7f09540b4cf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0968103b10 0x7f09681a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0968102fe0 0x7f0968199d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 --2- 172.21.15.50:0/3381529220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0968102600 0x7f0968199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.602+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 >> 172.21.15.50:0/3381529220 conn(0x7f09680fd010 msgr2=0x7f0968101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.603+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 shutdown_connections 2024-05-17T04:54:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:00.603+0000 7f096e111640 1 -- 172.21.15.50:0/3381529220 wait complete. 2024-05-17T04:54:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:02 smithi105 ceph-mon[34795]: pgmap v970: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-05-17T04:54:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:02 smithi110 ceph-mon[35497]: pgmap v970: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-05-17T04:54:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:02 smithi050 ceph-mon[31739]: pgmap v970: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 2.0 MiB/s wr, 299 op/s 2024-05-17T04:54:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:04 smithi105 ceph-mon[34795]: pgmap v971: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:54:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:04 smithi110 ceph-mon[35497]: pgmap v971: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:54:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:04 smithi050 ceph-mon[31739]: pgmap v971: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-05-17T04:54:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:06 smithi105 ceph-mon[34795]: pgmap v972: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-17T04:54:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:06 smithi110 ceph-mon[35497]: pgmap v972: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-17T04:54:06.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:06 smithi050 ceph-mon[31739]: pgmap v972: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.8 MiB/s wr, 357 op/s 2024-05-17T04:54:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:08 smithi105 ceph-mon[34795]: pgmap v973: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-05-17T04:54:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:08 smithi110 ceph-mon[35497]: pgmap v973: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-05-17T04:54:08.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:08 smithi050 ceph-mon[31739]: pgmap v973: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-05-17T04:54:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:10 smithi105 ceph-mon[34795]: pgmap v974: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-17T04:54:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:10 smithi110 ceph-mon[35497]: pgmap v974: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-17T04:54:10.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:10 smithi050 ceph-mon[31739]: pgmap v974: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 2.9 MiB/s wr, 337 op/s 2024-05-17T04:54:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:12 smithi105 ceph-mon[34795]: pgmap v975: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-17T04:54:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:12 smithi110 ceph-mon[35497]: pgmap v975: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-17T04:54:12.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:12 smithi050 ceph-mon[31739]: pgmap v975: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 2.5 MiB/s wr, 312 op/s 2024-05-17T04:54:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.745+0000 7fe91d354640 1 -- 172.21.15.50:0/2943564637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918103580 msgr2=0x7fe918103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.745+0000 7fe91d354640 1 --2- 172.21.15.50:0/2943564637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918103580 0x7fe918103990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe908009900 tx=0x7fe90802a410 comp rx=0 tx=0).stop 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 -- 172.21.15.50:0/2943564637 shutdown_connections 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 --2- 172.21.15.50:0/2943564637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918102bd0 0x7fe91810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 --2- 172.21.15.50:0/2943564637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 0x7fe918102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 --2- 172.21.15.50:0/2943564637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918103580 0x7fe918103990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 -- 172.21.15.50:0/2943564637 >> 172.21.15.50:0/2943564637 conn(0x7fe9180fd010 msgr2=0x7fe9180ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 -- 172.21.15.50:0/2943564637 shutdown_connections 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 -- 172.21.15.50:0/2943564637 wait complete. 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.746+0000 7fe91d354640 1 Processor -- start 2024-05-17T04:54:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 -- start start 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 0x7fe91819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 0x7fe91819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810dac0 con 0x7fe918103580 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810d940 con 0x7fe918102bd0 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe91d354640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe91810dc40 con 0x7fe918102280 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40982/0 (socket says 172.21.15.50:40982) 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 -- 172.21.15.50:0/3440357473 learned_addr learned my addr 172.21.15.50:0/3440357473 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 0x7fe91819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe916ffd640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 0x7fe91819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 -- 172.21.15.50:0/3440357473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 msgr2=0x7fe91819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 0x7fe91819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 -- 172.21.15.50:0/3440357473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 msgr2=0x7fe91819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 0x7fe91819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe9177fe640 1 -- 172.21.15.50:0/3440357473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9181a3590 con 0x7fe918103580 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.747+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 0x7fe91819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe9177fe640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe910009c90 tx=0x7fe91000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe914ff9640 1 -- 172.21.15.50:0/3440357473 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910003b70 con 0x7fe918103580 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9181a3880 con 0x7fe918103580 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe914ff9640 1 -- 172.21.15.50:0/3440357473 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe910003d10 con 0x7fe918103580 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe914ff9640 1 -- 172.21.15.50:0/3440357473 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910010790 con 0x7fe918103580 2024-05-17T04:54:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.748+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a3dd0 con 0x7fe918103580 2024-05-17T04:54:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.751+0000 7fe92420e740 1 Processor -- start 2024-05-17T04:54:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.752+0000 7fe914ff9640 1 -- 172.21.15.50:0/3440357473 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe910004020 con 0x7fe918103580 2024-05-17T04:54:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.752+0000 7fe92420e740 1 -- start start 2024-05-17T04:54:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 0x561c1e279cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1deb54d0 0x561c1e280c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561c1e1ebcc0 con 0x561c1deb54d0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c1e1db860 con 0x561c1dead2f0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe92420e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c1e1df8a0 con 0x561c1ded61b0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe916ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 0x561c1e279cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe90ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe90ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:40300/0 (socket says 172.21.15.50:40300) 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe90ffff640 1 -- 172.21.15.50:0/3284711164 learned_addr learned my addr 172.21.15.50:0/3284711164 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.753+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1deb54d0 0x561c1e280c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 -- 172.21.15.50:0/3284711164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 msgr2=0x561c1e279770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 -- 172.21.15.50:0/3284711164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1deb54d0 msgr2=0x561c1e280c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1deb54d0 0x561c1e280c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 -- 172.21.15.50:0/3284711164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1e1a3d50 con 0x561c1dead2f0 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe914ff9640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 0x7fe9040b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe916ffd640 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 0x561c1e279cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe908009230 tx=0x7fe908004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fefe69d5e63458 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe914ff9640 1 -- 172.21.15.50:0/3440357473 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9100e2b70 con 0x7fe918103580 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe90d7fa640 1 -- 172.21.15.50:0/3284711164 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908035030 con 0x561c1dead2f0 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe90d7fa640 1 -- 172.21.15.50:0/3284711164 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe908040070 con 0x561c1dead2f0 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.754+0000 7fe90d7fa640 1 -- 172.21.15.50:0/3284711164 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908031830 con 0x561c1dead2f0 2024-05-17T04:54:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.755+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 0x7fe9040b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- 172.21.15.50:0/3284711164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 msgr2=0x561c1e279cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe90ffff640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 0x7fe9040b4df0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x561c1e27a960 tx=0x7fe900005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 0x561c1e279cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe908009230 tx=0x7fe908004040 comp rx=0 tx=0).stop 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- 172.21.15.50:0/3284711164 shutdown_connections 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1deb54d0 0x561c1e280c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1dead2f0 0x561c1e279cf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 --2- 172.21.15.50:0/3284711164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e279770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- 172.21.15.50:0/3284711164 >> 172.21.15.50:0/3284711164 conn(0x561c1deca300 msgr2=0x561c1e2751a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- 172.21.15.50:0/3284711164 shutdown_connections 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- 172.21.15.50:0/3284711164 wait complete. 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 -- start start 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.756+0000 7fe92420e740 1 Processor -- start 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e283510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e28a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561c1e117ab0 con 0x561c1dead2f0 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c1e1984d0 con 0x561c1deb54d0 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe916ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c1e281630 con 0x561c1ded61b0 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe916ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40994/0 (socket says 172.21.15.50:40994) 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe916ffd640 1 -- 172.21.15.50:0/697027590 learned_addr learned my addr 172.21.15.50:0/697027590 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe90ffff640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e283510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe92420e740 10 client.0 _ll_register_callbacks cb 0x561c1e1d6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.757+0000 7fe9177fe640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e28a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 -- 172.21.15.50:0/697027590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 msgr2=0x561c1e28a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e28a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 -- 172.21.15.50:0/697027590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 msgr2=0x561c1e282fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 -- 172.21.15.50:0/697027590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1e199340 con 0x561c1deb54d0 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe9177fe640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e28a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe916ffd640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe90ffff640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e283510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fd0 tx=0x7fe900009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aca641dafbddfb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe8fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900005020 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe908040070 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9000051c0 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe900034050 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe92420e740 10 client.26453 fetch_fsmap learned FSMap version 43 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.758+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561c1df41790 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe908040070 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe90001e6e0 con 0x561c1deb54d0 2024-05-17T04:54:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 10 client.26453 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 20 client.26453 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de94d80 0x561c1e2908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 4 client.26453 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1e1edc10 con 0x561c1de94d80 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.759+0000 7fe916ffd640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de94d80 0x561c1e2908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.760+0000 7fe8fe7fc640 10 client.26453 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.760+0000 7fe916ffd640 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de94d80 0x561c1e2908f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0b5a806a984910 server_cookie=2002f544c45d976f in_seq=0 out_seq=0 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.760+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/697027590 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c1e1edc10 con 0x561c1de94d80 2024-05-17T04:54:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.760+0000 7fe8fe7fc640 10 client.26453 handle_command_reply: tid=0 2024-05-17T04:54:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 client.26453 shutdown 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 msgr2=0x561c1e283510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e283510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fd0 tx=0x7fe900009b80 comp rx=0 tx=0).stop 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 shutdown_connections 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e28a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e283510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de94d80 0x561c1e2908f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 --2- 172.21.15.50:0/697027590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e282fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 >> 172.21.15.50:0/697027590 conn(0x561c1deca300 msgr2=0x561c1e278010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe8fe7fc640 0 client.26453 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 shutdown_connections 2024-05-17T04:54:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 1 -- 172.21.15.50:0/697027590 wait complete. 2024-05-17T04:54:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.761+0000 7fe92420e740 20 client.26453 trim_cache size 0 max 16384 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 Processor -- start 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 -- start start 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e27db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e2785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181b0ac0 con 0x561c1dead2f0 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181ae390 con 0x561c1deb54d0 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe92420e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181ac1e0 con 0x561c1ded61b0 2024-05-17T04:54:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe916ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e27db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe90ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe90ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34328/0 (socket says 172.21.15.50:34328) 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.783+0000 7fe90ffff640 1 -- 172.21.15.50:0/2501226386 learned_addr learned my addr 172.21.15.50:0/2501226386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe9177fe640 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e2785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 -- 172.21.15.50:0/2501226386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 msgr2=0x561c1e2785b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e2785b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 -- 172.21.15.50:0/2501226386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 msgr2=0x561c1e27db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e27db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 -- 172.21.15.50:0/2501226386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1dfba620 con 0x561c1deb54d0 2024-05-17T04:54:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90ffff640 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe900021fd0 tx=0x7fe900005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe852d2938d6c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90cff9640 1 -- 172.21.15.50:0/2501226386 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90001e8e0 con 0x561c1deb54d0 2024-05-17T04:54:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90cff9640 1 -- 172.21.15.50:0/2501226386 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9000044d0 con 0x561c1deb54d0 2024-05-17T04:54:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.784+0000 7fe90cff9640 1 -- 172.21.15.50:0/2501226386 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900004730 con 0x561c1deb54d0 2024-05-17T04:54:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 -- 172.21.15.50:0/2501226386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 msgr2=0x561c1e290860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe900021fd0 tx=0x7fe900005e50 comp rx=0 tx=0).stop 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 -- 172.21.15.50:0/2501226386 shutdown_connections 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1ded61b0 0x561c1e2785b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1deb54d0 0x561c1e290860 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 --2- 172.21.15.50:0/2501226386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e27db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.785+0000 7fe92420e740 1 -- 172.21.15.50:0/2501226386 >> 172.21.15.50:0/2501226386 conn(0x561c1deca300 msgr2=0x561c1e277ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.786+0000 7fe92420e740 1 -- 172.21.15.50:0/2501226386 shutdown_connections 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.786+0000 7fe92420e740 1 -- 172.21.15.50:0/2501226386 wait complete. 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.786+0000 7fe92420e740 1 -- start start 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.786+0000 7fe92420e740 1 Processor -- start 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e28ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 0x561c1e2891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181ac850 con 0x561c1dead2f0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181adcc0 con 0x561c1ded61b0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9181acf20 con 0x561c1deb54d0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe92420e740 10 client.0 _ll_register_callbacks cb 0x7fe900009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:34338/0 (socket says 172.21.15.50:34338) 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 -- 172.21.15.50:0/1642906385 learned_addr learned my addr 172.21.15.50:0/1642906385 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe916ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e28ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe90ffff640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 0x561c1e2891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 -- 172.21.15.50:0/1642906385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 msgr2=0x561c1e2891c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 0x561c1e2891c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 -- 172.21.15.50:0/1642906385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 msgr2=0x561c1e28ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e28ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 -- 172.21.15.50:0/1642906385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1e289ed0 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe916ffd640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e28ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe90ffff640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 0x561c1e2891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe9177fe640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9100e0280 tx=0x7fe9100046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a8d51dd3dfc0be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.787+0000 7fe8fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100c9060 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9100a9210 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9000044d0 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe9100a98b0 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 10 client.26459 fetch_fsmap learned FSMap version 43 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561c1e300500 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9000044d0 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fe9100a8a70 con 0x561c1ded61b0 2024-05-17T04:54:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 10 client.26459 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 20 client.26459 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de6a1c0 0x561c1e302ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 4 client.26459 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.788+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c1e1e73f0 con 0x561c1de6a1c0 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.789+0000 7fe916ffd640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de6a1c0 0x561c1e302ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.789+0000 7fe8fe7fc640 10 client.26459 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.789+0000 7fe916ffd640 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de6a1c0 0x561c1e302ea0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e751a098a2ca34fe server_cookie=dc2b414bc6ee364c in_seq=0 out_seq=0 2024-05-17T04:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe8fe7fc640 1 -- 172.21.15.50:0/1642906385 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561c1e1e73f0 con 0x561c1de6a1c0 2024-05-17T04:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe8fe7fc640 10 client.26459 handle_command_reply: tid=0 2024-05-17T04:54:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 client.26459 shutdown 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 msgr2=0x561c1e289740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9100e0280 tx=0x7fe9100046e0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 shutdown_connections 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1ded61b0 0x561c1e289740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c1deb54d0 0x561c1e2891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x561c1de6a1c0 0x561c1e302ea0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 --2- 172.21.15.50:0/1642906385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561c1dead2f0 0x561c1e28ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe8fe7fc640 0 client.26459 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 >> 172.21.15.50:0/1642906385 conn(0x561c1deca300 msgr2=0x561c1e27cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 shutdown_connections 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.805+0000 7fe92420e740 1 -- 172.21.15.50:0/1642906385 wait complete. 2024-05-17T04:54:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe92420e740 20 client.26459 trim_cache size 0 max 16384 2024-05-17T04:54:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:54:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:54:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 msgr2=0x7fe9040b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 0x7fe9040b4df0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x561c1e27a960 tx=0x7fe900005f90 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 msgr2=0x7fe9181a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe910009c90 tx=0x7fe91000c6a0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 shutdown_connections 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fe9040b28e0 0x7fe9040b4df0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe918103580 0x7fe9181a2f20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe918102bd0 0x7fe91819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 --2- 172.21.15.50:0/3440357473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe918102280 0x7fe91819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.806+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 >> 172.21.15.50:0/3440357473 conn(0x7fe9180fd010 msgr2=0x7fe9180feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.807+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 shutdown_connections 2024-05-17T04:54:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:13.807+0000 7fe91d354640 1 -- 172.21.15.50:0/3440357473 wait complete. 2024-05-17T04:54:14.566 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:14 smithi105 ceph-mon[34795]: pgmap v976: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:54:14.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:14 smithi110 ceph-mon[35497]: pgmap v976: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:54:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:14 smithi050 ceph-mon[31739]: pgmap v976: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.5 MiB/s wr, 303 op/s 2024-05-17T04:54:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:16 smithi105 ceph-mon[34795]: pgmap v977: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:54:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:16 smithi110 ceph-mon[35497]: pgmap v977: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:54:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:16 smithi050 ceph-mon[31739]: pgmap v977: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 2.4 MiB/s wr, 284 op/s 2024-05-17T04:54:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:18 smithi105 ceph-mon[34795]: pgmap v978: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-17T04:54:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:18 smithi110 ceph-mon[35497]: pgmap v978: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-17T04:54:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:18 smithi050 ceph-mon[31739]: pgmap v978: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-05-17T04:54:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:20 smithi105 ceph-mon[34795]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-05-17T04:54:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:20 smithi110 ceph-mon[35497]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-05-17T04:54:20.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:20 smithi050 ceph-mon[31739]: pgmap v979: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-05-17T04:54:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:22 smithi105 ceph-mon[34795]: pgmap v980: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T04:54:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:22 smithi110 ceph-mon[35497]: pgmap v980: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T04:54:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:22 smithi050 ceph-mon[31739]: pgmap v980: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-17T04:54:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:24 smithi105 ceph-mon[34795]: pgmap v981: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-05-17T04:54:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:24 smithi110 ceph-mon[35497]: pgmap v981: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-05-17T04:54:24.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:24 smithi050 ceph-mon[31739]: pgmap v981: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-05-17T04:54:25.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:25 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:25.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:25 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:25.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:25 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:26 smithi105 ceph-mon[34795]: pgmap v982: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-17T04:54:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:26 smithi110 ceph-mon[35497]: pgmap v982: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-17T04:54:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:26 smithi050 ceph-mon[31739]: pgmap v982: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-05-17T04:54:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 -- 172.21.15.50:0/3771537054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 msgr2=0x7fa28c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 --2- 172.21.15.50:0/3771537054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c1035e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa280009900 tx=0x7fa28002a410 comp rx=0 tx=0).stop 2024-05-17T04:54:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 -- 172.21.15.50:0/3771537054 shutdown_connections 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 --2- 172.21.15.50:0/3771537054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 0x7fa28c10a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 --2- 172.21.15.50:0/3771537054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c1035e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 --2- 172.21.15.50:0/3771537054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 0x7fa28c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 -- 172.21.15.50:0/3771537054 >> 172.21.15.50:0/3771537054 conn(0x7fa28c0fd8b0 msgr2=0x7fa28c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 -- 172.21.15.50:0/3771537054 shutdown_connections 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.933+0000 7fa291e4d640 1 -- 172.21.15.50:0/3771537054 wait complete. 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 Processor -- start 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 -- start start 2024-05-17T04:54:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 0x7fa28c10c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 0x7fa28c10b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10eba0 con 0x7fa28c1031d0 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10ea20 con 0x7fa28c1020b0 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa291e4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10ed20 con 0x7fa28c103b20 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa28affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa28affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36568/0 (socket says 172.21.15.50:36568) 2024-05-17T04:54:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa28affd640 1 -- 172.21.15.50:0/1173638228 learned_addr learned my addr 172.21.15.50:0/1173638228 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 0x7fa28c10c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.934+0000 7fa28bfff640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 0x7fa28c10b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 -- 172.21.15.50:0/1173638228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 msgr2=0x7fa28c10b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 0x7fa28c10b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 -- 172.21.15.50:0/1173638228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 msgr2=0x7fa28c10c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 0x7fa28c10c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 -- 172.21.15.50:0/1173638228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c10bca0 con 0x7fa28c1031d0 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa28affd640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa280009230 tx=0x7fa280004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa288ff9640 1 -- 172.21.15.50:0/1173638228 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28003f030 con 0x7fa28c1031d0 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa288ff9640 1 -- 172.21.15.50:0/1173638228 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa280004440 con 0x7fa28c1031d0 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a38b0 con 0x7fa28c1031d0 2024-05-17T04:54:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa288ff9640 1 -- 172.21.15.50:0/1173638228 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280035070 con 0x7fa28c1031d0 2024-05-17T04:54:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.935+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a3cb0 con 0x7fa28c1031d0 2024-05-17T04:54:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 Processor -- start 2024-05-17T04:54:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 -- start start 2024-05-17T04:54:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa288ff9640 1 -- 172.21.15.50:0/1173638228 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa280042030 con 0x7fa28c1031d0 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 0x5593cf235770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee714d0 0x5593cf23cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5593cf1a7cc0 con 0x5593cee692f0 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5593cf197860 con 0x5593cee921b0 2024-05-17T04:54:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa298ddf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5593cf19b8a0 con 0x5593cee714d0 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36584/0 (socket says 172.21.15.50:36584) 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28b7fe640 1 -- 172.21.15.50:0/4006754167 learned_addr learned my addr 172.21.15.50:0/4006754167 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 0x5593cf235770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28affd640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee714d0 0x5593cf23cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 -- 172.21.15.50:0/4006754167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee714d0 msgr2=0x5593cf23cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee714d0 0x5593cf23cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 -- 172.21.15.50:0/4006754167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 msgr2=0x5593cf235cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28bfff640 1 -- 172.21.15.50:0/4006754167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593cf15fd50 con 0x5593cee921b0 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.938+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa28bfff640 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 0x5593cf235770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f00 tx=0x7fa284011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ae4b2b9cdaa14f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa288ff9640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 0x7fa2740b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa271ffb640 1 -- 172.21.15.50:0/4006754167 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284004d20 con 0x5593cee921b0 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa271ffb640 1 -- 172.21.15.50:0/4006754167 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa284004030 con 0x5593cee921b0 2024-05-17T04:54:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa271ffb640 1 -- 172.21.15.50:0/4006754167 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284005280 con 0x5593cee921b0 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 0x7fa2740b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.939+0000 7fa288ff9640 1 -- 172.21.15.50:0/1173638228 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa2801029d0 con 0x7fa28c1031d0 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 0x7fa2740b4cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5593cf236b70 tx=0x7fa27c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- 172.21.15.50:0/4006754167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 msgr2=0x5593cf235770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 0x5593cf235770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f00 tx=0x7fa284011810 comp rx=0 tx=0).stop 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- 172.21.15.50:0/4006754167 shutdown_connections 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee714d0 0x5593cf23cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf235cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4006754167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee921b0 0x5593cf235770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- 172.21.15.50:0/4006754167 >> 172.21.15.50:0/4006754167 conn(0x5593cee86300 msgr2=0x5593cf2311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- 172.21.15.50:0/4006754167 shutdown_connections 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- 172.21.15.50:0/4006754167 wait complete. 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 -- start start 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.940+0000 7fa298ddf740 1 Processor -- start 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf2465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5593cf0d3ab0 con 0x5593cee692f0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5593cf1544d0 con 0x5593cee714d0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5593cf23d690 con 0x5593cee921b0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa298ddf740 10 client.0 _ll_register_callbacks cb 0x5593cf192950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36594/0 (socket says 172.21.15.50:36594) 2024-05-17T04:54:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf2465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 -- 172.21.15.50:0/2625603309 learned_addr learned my addr 172.21.15.50:0/2625603309 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:37860/0 (socket says 172.21.15.50:37860) 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 -- 172.21.15.50:0/2625603309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 msgr2=0x5593cf2465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf2465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 -- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 msgr2=0x5593cf23f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 -- 172.21.15.50:0/2625603309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cf155340 con 0x5593cee692f0 2024-05-17T04:54:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28affd640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28bfff640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf2465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004570 tx=0x7fa27c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17069f3f0b679e7b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.941+0000 7fa26affd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0099b0 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa284004030 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa27c00f690 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa27c00f890 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 10 client.16719 fetch_fsmap learned FSMap version 44 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5593ceefd790 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa284004030 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fa27c027070 con 0x5593cee692f0 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 10 client.16719 fetch_fsmap finished waiting for FSMap version 44 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 20 client.16719 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee50d80 0x5593cf24c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 4 client.16719 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593cf1a9c10 con 0x5593cee50d80 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.942+0000 7fa28affd640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee50d80 0x5593cf24c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.943+0000 7fa26affd640 10 client.16719 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.943+0000 7fa28affd640 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee50d80 0x5593cf24c9f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b96ce0ca50aa1e6 server_cookie=67d029ca879066c4 in_seq=0 out_seq=0 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa26affd640 1 -- 172.21.15.50:0/2625603309 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593cf1a9c10 con 0x5593cee50d80 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa26affd640 10 client.16719 handle_command_reply: tid=0 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 client.16719 shutdown 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 msgr2=0x5593cf23f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa27c004570 tx=0x7fa27c004730 comp rx=0 tx=0).stop 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 shutdown_connections 2024-05-17T04:54:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf2465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf23f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee50d80 0x5593cf24c9f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 --2- 172.21.15.50:0/2625603309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee692f0 0x5593cf23f120 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 >> 172.21.15.50:0/2625603309 conn(0x5593cee86300 msgr2=0x5593cf2301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa26affd640 0 client.16719 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 shutdown_connections 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 1 -- 172.21.15.50:0/2625603309 wait complete. 2024-05-17T04:54:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.944+0000 7fa298ddf740 20 client.16719 trim_cache size 0 max 16384 2024-05-17T04:54:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 Processor -- start 2024-05-17T04:54:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 -- start start 2024-05-17T04:54:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee692f0 0x5593cf231270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf2464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1b0a40 con 0x5593cee921b0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1ae2e0 con 0x5593cee714d0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa298ddf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1ac130 con 0x5593cee692f0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36602/0 (socket says 172.21.15.50:36602) 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee692f0 0x5593cf231270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 -- 172.21.15.50:0/600386354 learned_addr learned my addr 172.21.15.50:0/600386354 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28affd640 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf2464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 -- 172.21.15.50:0/600386354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee692f0 msgr2=0x5593cf231270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee692f0 0x5593cf231270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 -- 172.21.15.50:0/600386354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 msgr2=0x5593cf2464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf2464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 -- 172.21.15.50:0/600386354 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593cef76620 con 0x5593cee921b0 2024-05-17T04:54:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.960+0000 7fa28bfff640 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x5593cf236960 tx=0x7fa284005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6009b8c8bc623ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa2717fa640 1 -- 172.21.15.50:0/600386354 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284028070 con 0x5593cee921b0 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa2717fa640 1 -- 172.21.15.50:0/600386354 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa284004040 con 0x5593cee921b0 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa2717fa640 1 -- 172.21.15.50:0/600386354 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284005280 con 0x5593cee921b0 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 -- 172.21.15.50:0/600386354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 msgr2=0x5593cf246a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x5593cf236960 tx=0x7fa284005bd0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 -- 172.21.15.50:0/600386354 shutdown_connections 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee921b0 0x5593cf246a20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee714d0 0x5593cf2464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 --2- 172.21.15.50:0/600386354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee692f0 0x5593cf231270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.961+0000 7fa298ddf740 1 -- 172.21.15.50:0/600386354 >> 172.21.15.50:0/600386354 conn(0x5593cee86300 msgr2=0x5593cf230b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- 172.21.15.50:0/600386354 shutdown_connections 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- 172.21.15.50:0/600386354 wait complete. 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- start start 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 Processor -- start 2024-05-17T04:54:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee692f0 0x5593cf246780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf243360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1ac7a0 con 0x5593cee714d0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1adc10 con 0x5593cee692f0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1ace70 con 0x5593cee921b0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa298ddf740 10 client.0 _ll_register_callbacks cb 0x7fa27c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa28affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa28affd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36608/0 (socket says 172.21.15.50:36608) 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa28affd640 1 -- 172.21.15.50:0/4021962651 learned_addr learned my addr 172.21.15.50:0/4021962651 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee692f0 0x5593cf246780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.962+0000 7fa28bfff640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf243360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 -- 172.21.15.50:0/4021962651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 msgr2=0x5593cf243360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf243360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 -- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee692f0 msgr2=0x5593cf246780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee692f0 0x5593cf246780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 -- 172.21.15.50:0/4021962651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cf243af0 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa28affd640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa280100140 tx=0x7fa280100010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57a580fb070be08 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280049040 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa2800c9f70 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa284004040 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa2800d2400 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa298ddf740 10 client.16725 fetch_fsmap learned FSMap version 44 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5593cf2b3b70 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa284004040 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.963+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fa2800d2630 con 0x5593cee714d0 2024-05-17T04:54:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 10 client.16725 fetch_fsmap finished waiting for FSMap version 44 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 20 client.16725 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee261c0 0x5593cf2b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 4 client.16725 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593cf1a33f0 con 0x5593cee261c0 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee261c0 0x5593cf2b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa26affd640 10 client.16725 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:26.964+0000 7fa28b7fe640 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee261c0 0x5593cf2b64e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe308299cd0fa1f server_cookie=b2b63ee3eadf9e69 in_seq=0 out_seq=0 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa26affd640 1 -- 172.21.15.50:0/4021962651 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5593cf1a33f0 con 0x5593cee261c0 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa26affd640 10 client.16725 handle_command_reply: tid=0 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 client.16725 shutdown 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 msgr2=0x5593cf246cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa280100140 tx=0x7fa280100010 comp rx=0 tx=0).stop 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 shutdown_connections 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5593cee921b0 0x5593cf243360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x5593cee261c0 0x5593cf2b64e0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5593cee714d0 0x5593cf246cc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 --2- 172.21.15.50:0/4021962651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593cee692f0 0x5593cf246780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 >> 172.21.15.50:0/4021962651 conn(0x5593cee86300 msgr2=0x5593cf22f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa26affd640 0 client.16725 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 shutdown_connections 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 1 -- 172.21.15.50:0/4021962651 wait complete. 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.078+0000 7fa298ddf740 20 client.16725 trim_cache size 0 max 16384 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:54:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 msgr2=0x7fa2740b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 0x7fa2740b4cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5593cf236b70 tx=0x7fa27c005f90 comp rx=0 tx=0).stop 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 msgr2=0x7fa28c10b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa280009230 tx=0x7fa280004040 comp rx=0 tx=0).stop 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 shutdown_connections 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa28c103b20 0x7fa28c10b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fa2740b27e0 0x7fa2740b4cf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa28c1031d0 0x7fa28c10b030 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 --2- 172.21.15.50:0/1173638228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa28c1020b0 0x7fa28c10c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 >> 172.21.15.50:0/1173638228 conn(0x7fa28c0fd8b0 msgr2=0x7fa28c106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 shutdown_connections 2024-05-17T04:54:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:27.079+0000 7fa291e4d640 1 -- 172.21.15.50:0/1173638228 wait complete. 2024-05-17T04:54:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:28 smithi105 ceph-mon[34795]: pgmap v983: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 941 KiB/s wr, 124 op/s 2024-05-17T04:54:28.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:28 smithi110 ceph-mon[35497]: pgmap v983: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 941 KiB/s wr, 124 op/s 2024-05-17T04:54:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:28 smithi050 ceph-mon[31739]: pgmap v983: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 941 KiB/s wr, 124 op/s 2024-05-17T04:54:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:30 smithi105 ceph-mon[34795]: pgmap v984: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-17T04:54:30.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:30 smithi105 ceph-mon[34795]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:30 smithi110 ceph-mon[35497]: pgmap v984: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-17T04:54:30.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:30 smithi110 ceph-mon[35497]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:30 smithi050 ceph-mon[31739]: pgmap v984: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-17T04:54:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:30 smithi050 ceph-mon[31739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-17T04:54:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:32 smithi105 ceph-mon[34795]: pgmap v985: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:32 smithi110 ceph-mon[35497]: pgmap v985: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:32 smithi050 ceph-mon[31739]: pgmap v985: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:34.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:34 smithi105 ceph-mon[34795]: pgmap v986: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-05-17T04:54:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:34 smithi110 ceph-mon[35497]: pgmap v986: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-05-17T04:54:34.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:34 smithi050 ceph-mon[31739]: pgmap v986: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-05-17T04:54:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:36 smithi105 ceph-mon[34795]: pgmap v987: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-05-17T04:54:36.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:36 smithi110 ceph-mon[35497]: pgmap v987: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-05-17T04:54:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:36 smithi050 ceph-mon[31739]: pgmap v987: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 203 op/s 2024-05-17T04:54:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:38 smithi105 ceph-mon[34795]: pgmap v988: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-17T04:54:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:38 smithi110 ceph-mon[35497]: pgmap v988: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-17T04:54:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:38 smithi050 ceph-mon[31739]: pgmap v988: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 -- 172.21.15.50:0/663724835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 msgr2=0x7fc02c10b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 --2- 172.21.15.50:0/663724835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c10b360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc024009be0 tx=0x7fc02402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 -- 172.21.15.50:0/663724835 shutdown_connections 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 --2- 172.21.15.50:0/663724835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c10b360 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 --2- 172.21.15.50:0/663724835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 0x7fc02c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 --2- 172.21.15.50:0/663724835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 -- 172.21.15.50:0/663724835 >> 172.21.15.50:0/663724835 conn(0x7fc02c0fd8b0 msgr2=0x7fc02c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 -- 172.21.15.50:0/663724835 shutdown_connections 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.223+0000 7fc031072640 1 -- 172.21.15.50:0/663724835 wait complete. 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 Processor -- start 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 -- start start 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 0x7fc02c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10eb60 con 0x7fc02c103c80 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10e9e0 con 0x7fc02c1065e0 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc031072640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c10ece0 con 0x7fc02c108f40 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33220/0 (socket says 172.21.15.50:33220) 2024-05-17T04:54:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc02ad9d640 1 -- 172.21.15.50:0/1458706990 learned_addr learned my addr 172.21.15.50:0/1458706990 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.224+0000 7fc023fff640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 0x7fc02c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 -- 172.21.15.50:0/1458706990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 msgr2=0x7fc02c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 -- 172.21.15.50:0/1458706990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 msgr2=0x7fc02c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 0x7fc02c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 -- 172.21.15.50:0/1458706990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c1a12c0 con 0x7fc02c103c80 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00e9f0 tx=0x7fc01c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc028d99640 1 -- 172.21.15.50:0/1458706990 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ccc0 con 0x7fc02c103c80 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc028d99640 1 -- 172.21.15.50:0/1458706990 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc01c0035f0 con 0x7fc02c103c80 2024-05-17T04:54:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c1a15b0 con 0x7fc02c103c80 2024-05-17T04:54:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc028d99640 1 -- 172.21.15.50:0/1458706990 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ce60 con 0x7fc02c103c80 2024-05-17T04:54:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.225+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c1a1b00 con 0x7fc02c103c80 2024-05-17T04:54:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.228+0000 7fc028d99640 1 -- 172.21.15.50:0/1458706990 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc01c004000 con 0x7fc02c103c80 2024-05-17T04:54:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 Processor -- start 2024-05-17T04:54:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 -- start start 2024-05-17T04:54:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc028d99640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 0x7fc0180b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd36770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd3dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a20fca8cc0 con 0x55a20f9724d0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a20fc98860 con 0x55a20f96a2f0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc03800d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a20fc9c8a0 con 0x55a20f9931b0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc028d99640 1 -- 172.21.15.50:0/1458706990 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc01c0e27d0 con 0x7fc02c103c80 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 0x7fc0180b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc02b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd3dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.229+0000 7fc023fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:40284/0 (socket says 172.21.15.50:40284) 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 -- 172.21.15.50:0/1469004032 learned_addr learned my addr 172.21.15.50:0/1469004032 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc02b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd3dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33228/0 (socket says 172.21.15.50:33228) 2024-05-17T04:54:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd36770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 -- 172.21.15.50:0/1469004032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 msgr2=0x55a20fd36770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd36770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 -- 172.21.15.50:0/1469004032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 msgr2=0x55a20fd3dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd3dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 -- 172.21.15.50:0/1469004032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20fc60d50 con 0x55a20f96a2f0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc023fff640 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc014009c80 tx=0x7fc014011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d9b0920448759c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 0x7fc0180b4cf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc024030800 tx=0x7fc0240098d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc0217fa640 1 -- 172.21.15.50:0/1469004032 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01401e070 con 0x55a20f96a2f0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc0217fa640 1 -- 172.21.15.50:0/1469004032 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc014019ac0 con 0x55a20f96a2f0 2024-05-17T04:54:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.230+0000 7fc0217fa640 1 -- 172.21.15.50:0/1469004032 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01400e840 con 0x55a20f96a2f0 2024-05-17T04:54:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 -- 172.21.15.50:0/1469004032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 msgr2=0x55a20fd36cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc014009c80 tx=0x7fc014011810 comp rx=0 tx=0).stop 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 -- 172.21.15.50:0/1469004032 shutdown_connections 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd3dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd36cf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 --2- 172.21.15.50:0/1469004032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd36770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 -- 172.21.15.50:0/1469004032 >> 172.21.15.50:0/1469004032 conn(0x55a20f987300 msgr2=0x55a20fd321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 -- 172.21.15.50:0/1469004032 shutdown_connections 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.231+0000 7fc03800d740 1 -- 172.21.15.50:0/1469004032 wait complete. 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.232+0000 7fc03800d740 1 -- start start 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.232+0000 7fc03800d740 1 Processor -- start 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.232+0000 7fc03800d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.232+0000 7fc03800d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.232+0000 7fc03800d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd4b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc03800d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a20fbd4ab0 con 0x55a20f9724d0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc023fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc023fff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33242/0 (socket says 172.21.15.50:33242) 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc023fff640 1 -- 172.21.15.50:0/2849041626 learned_addr learned my addr 172.21.15.50:0/2849041626 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a20fc554d0 con 0x55a20f96a2f0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a20fd42aa0 con 0x55a20f9931b0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc03800d740 10 client.0 _ll_register_callbacks cb 0x55a20fc93950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02b59e640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd4b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 -- 172.21.15.50:0/2849041626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 msgr2=0x55a20fd4b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd4b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 -- 172.21.15.50:0/2849041626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 msgr2=0x55a20fd449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20fc56340 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd44470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0dff90 tx=0x7fc01c0a51f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4296ec8e7aae560 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc0127fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0aa160 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc01c0b2430 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.233+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc014019ac0 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc01c0a9a60 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 10 client.26489 fetch_fsmap learned FSMap version 45 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a20f9fe790 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc014019ac0 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fc01c0a8e50 con 0x55a20f96a2f0 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 10 client.26489 fetch_fsmap finished waiting for FSMap version 45 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 20 client.26489 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f951d80 0x55a20fd51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 4 client.26489 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a20fcaac10 con 0x55a20f951d80 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.234+0000 7fc023fff640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f951d80 0x55a20fd51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.235+0000 7fc0127fc640 10 client.26489 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.235+0000 7fc023fff640 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f951d80 0x55a20fd51d00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c935d270b463e03 server_cookie=539a113e815d53eb in_seq=0 out_seq=0 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc0127fc640 1 -- 172.21.15.50:0/2849041626 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a20fcaac10 con 0x55a20f951d80 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc0127fc640 10 client.26489 handle_command_reply: tid=0 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 client.26489 shutdown 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 msgr2=0x55a20fd44470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd44470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0dff90 tx=0x7fc01c0a51f0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 shutdown_connections 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd4b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f951d80 0x55a20fd51d00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9724d0 0x55a20fd449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc0127fc640 0 client.26489 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 --2- 172.21.15.50:0/2849041626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fd44470 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 >> 172.21.15.50:0/2849041626 conn(0x55a20f987300 msgr2=0x55a20fd35010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 shutdown_connections 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 1 -- 172.21.15.50:0/2849041626 wait complete. 2024-05-17T04:54:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.236+0000 7fc03800d740 20 client.26489 trim_cache size 0 max 16384 2024-05-17T04:54:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 Processor -- start 2024-05-17T04:54:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 -- start start 2024-05-17T04:54:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f9724d0 0x55a20fd35500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1ac6f0 con 0x55a20f96a2f0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1a9f60 con 0x55a20f9724d0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc03800d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1a7db0 con 0x55a20f9931b0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc02b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc023fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f9724d0 0x55a20fd35500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc02b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57756/0 (socket says 172.21.15.50:57756) 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc02b59e640 1 -- 172.21.15.50:0/1060804083 learned_addr learned my addr 172.21.15.50:0/1060804083 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.253+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:33258/0 (socket says 172.21.15.50:33258) 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 -- 172.21.15.50:0/1060804083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f9724d0 msgr2=0x55a20fd35500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f9724d0 0x55a20fd35500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 -- 172.21.15.50:0/1060804083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 msgr2=0x55a20fd3ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 -- 172.21.15.50:0/1060804083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20fa77620 con 0x55a20f9931b0 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc02402acb0 tx=0x7fc02402a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345e22c84f036079 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc020ff9640 1 -- 172.21.15.50:0/1060804083 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024003b30 con 0x55a20f9931b0 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc020ff9640 1 -- 172.21.15.50:0/1060804083 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc024051070 con 0x55a20f9931b0 2024-05-17T04:54:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.254+0000 7fc020ff9640 1 -- 172.21.15.50:0/1060804083 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02404c6d0 con 0x55a20f9931b0 2024-05-17T04:54:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 -- 172.21.15.50:0/1060804083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 msgr2=0x55a20fd35a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc02402acb0 tx=0x7fc02402a6c0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 -- 172.21.15.50:0/1060804083 shutdown_connections 2024-05-17T04:54:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9931b0 0x55a20fd35a40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f9724d0 0x55a20fd35500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 --2- 172.21.15.50:0/1060804083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f96a2f0 0x55a20fd3ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 -- 172.21.15.50:0/1060804083 >> 172.21.15.50:0/1060804083 conn(0x55a20f987300 msgr2=0x55a20fd34df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 -- 172.21.15.50:0/1060804083 shutdown_connections 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.255+0000 7fc03800d740 1 -- 172.21.15.50:0/1060804083 wait complete. 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 -- start start 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 Processor -- start 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fdb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 0x55a20fdb8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1a8420 con 0x55a20f9931b0 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1a9890 con 0x55a20f96a2f0 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c1a8af0 con 0x55a20f9724d0 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc03800d740 10 client.0 _ll_register_callbacks cb 0x7fc01c0046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 0x55a20fdb8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc023fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fdb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc023fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:57770/0 (socket says 172.21.15.50:57770) 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc023fff640 1 -- 172.21.15.50:0/1597622399 learned_addr learned my addr 172.21.15.50:0/1597622399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 -- 172.21.15.50:0/1597622399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 msgr2=0x55a20fdb6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 -- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 msgr2=0x55a20fdb4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fdb4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc02b59e640 1 -- 172.21.15.50:0/1597622399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20fdb9120 con 0x55a20f9931b0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.256+0000 7fc023fff640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc02b59e640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 0x55a20fdb8ae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc024034040 tx=0x7fc024004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95b29ec3ad1f5c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc0127fc640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02405a070 con 0x55a20f9931b0 2024-05-17T04:54:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc02404c470 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc024051070 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc024068050 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc03800d740 10 client.16743 fetch_fsmap learned FSMap version 45 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a20fdba3b0 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.257+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc024051070 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fc02403c070 con 0x55a20f9931b0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 10 client.16743 fetch_fsmap finished waiting for FSMap version 45 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 20 client.16743 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f9271c0 0x55a20fdbcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 4 client.16743 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a20fca43f0 con 0x55a20f9271c0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f9271c0 0x55a20fdbcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc0127fc640 10 client.16743 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.258+0000 7fc02ad9d640 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f9271c0 0x55a20fdbcd80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523fb7d3617db89a server_cookie=a576f9c1b634ed59 in_seq=0 out_seq=0 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc0127fc640 1 -- 172.21.15.50:0/1597622399 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a20fca43f0 con 0x55a20f9271c0 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc0127fc640 10 client.16743 handle_command_reply: tid=0 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 client.16743 shutdown 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 msgr2=0x55a20fdb8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 0x55a20fdb8ae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc024034040 tx=0x7fc024004210 comp rx=0 tx=0).stop 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 shutdown_connections 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x55a20f9271c0 0x55a20fdbcd80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a20f9931b0 0x55a20fdb8ae0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a20f9724d0 0x55a20fdb6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 --2- 172.21.15.50:0/1597622399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a20f96a2f0 0x55a20fdb4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 >> 172.21.15.50:0/1597622399 conn(0x55a20f987300 msgr2=0x55a20fd357b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.268+0000 7fc0127fc640 0 client.16743 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 shutdown_connections 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc03800d740 1 -- 172.21.15.50:0/1597622399 wait complete. 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc03800d740 20 client.16743 trim_cache size 0 max 16384 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:54:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 msgr2=0x7fc0180b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 0x7fc0180b4cf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc024030800 tx=0x7fc0240098d0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 msgr2=0x7fc02c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00e9f0 tx=0x7fc01c00eeb0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 shutdown_connections 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc02c108f40 0x7fc02c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc02c1065e0 0x7fc02c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fc0180b27e0 0x7fc0180b4cf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 --2- 172.21.15.50:0/1458706990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc02c103c80 0x7fc02c199780 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 >> 172.21.15.50:0/1458706990 conn(0x7fc02c0fd8b0 msgr2=0x7fc02c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 shutdown_connections 2024-05-17T04:54:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:40.269+0000 7fc031072640 1 -- 172.21.15.50:0/1458706990 wait complete. 2024-05-17T04:54:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:40 smithi105 ceph-mon[34795]: pgmap v989: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:40.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:40 smithi110 ceph-mon[35497]: pgmap v989: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:40 smithi050 ceph-mon[31739]: pgmap v989: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-05-17T04:54:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:42 smithi105 ceph-mon[34795]: pgmap v990: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-17T04:54:42.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:42 smithi110 ceph-mon[35497]: pgmap v990: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-17T04:54:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:42 smithi050 ceph-mon[31739]: pgmap v990: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-05-17T04:54:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:44 smithi105 ceph-mon[34795]: pgmap v991: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:54:44.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:44 smithi110 ceph-mon[35497]: pgmap v991: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:54:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:44 smithi050 ceph-mon[31739]: pgmap v991: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-05-17T04:54:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:46 smithi105 ceph-mon[34795]: pgmap v992: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-17T04:54:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:46 smithi110 ceph-mon[35497]: pgmap v992: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-17T04:54:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:46 smithi050 ceph-mon[31739]: pgmap v992: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-05-17T04:54:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:48 smithi105 ceph-mon[34795]: pgmap v993: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-05-17T04:54:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:48 smithi110 ceph-mon[35497]: pgmap v993: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-05-17T04:54:48.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:48 smithi050 ceph-mon[31739]: pgmap v993: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-05-17T04:54:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:50 smithi105 ceph-mon[34795]: pgmap v994: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-17T04:54:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:50 smithi110 ceph-mon[35497]: pgmap v994: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-17T04:54:50.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:50 smithi050 ceph-mon[31739]: pgmap v994: 129 pgs: 129 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-17T04:54:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:52 smithi105 ceph-mon[34795]: pgmap v995: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-17T04:54:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:52 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:54:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:52 smithi110 ceph-mon[35497]: pgmap v995: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-17T04:54:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:52 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:54:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:52 smithi050 ceph-mon[31739]: pgmap v995: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-05-17T04:54:52.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:52 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:54:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 -- 172.21.15.50:0/2648599527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 msgr2=0x7f713410d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 --2- 172.21.15.50:0/2648599527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f713410d700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f712c009ca0 tx=0x7f712c02a8c0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 -- 172.21.15.50:0/2648599527 shutdown_connections 2024-05-17T04:54:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 --2- 172.21.15.50:0/2648599527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f713410d700 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 --2- 172.21.15.50:0/2648599527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71341064b0 0x7f71341068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 --2- 172.21.15.50:0/2648599527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7134105390 0x7f71341057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 -- 172.21.15.50:0/2648599527 >> 172.21.15.50:0/2648599527 conn(0x7f7134100b70 msgr2=0x7f7134103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 -- 172.21.15.50:0/2648599527 shutdown_connections 2024-05-17T04:54:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.403+0000 7f713beac640 1 -- 172.21.15.50:0/2648599527 wait complete. 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 Processor -- start 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 -- start start 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7134105390 0x7f713419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 0x7f713419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7134110d20 con 0x7f7134106e00 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7134110ba0 con 0x7f71341064b0 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713beac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7134110ea0 con 0x7f7134105390 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59254/0 (socket says 172.21.15.50:59254) 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f713a449640 1 -- 172.21.15.50:0/4026021712 learned_addr learned my addr 172.21.15.50:0/4026021712 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.404+0000 7f7139447640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 0x7f713419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139c48640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7134105390 0x7f713419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 -- 172.21.15.50:0/4026021712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7134105390 msgr2=0x7f713419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7134105390 0x7f713419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 -- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 msgr2=0x7f71341a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 -- 172.21.15.50:0/4026021712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a34d0 con 0x7f71341064b0 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f713a449640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f7139447640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 0x7f713419bed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9f0 tx=0x7f712400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f712affd640 1 -- 172.21.15.50:0/4026021712 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400ccc0 con 0x7f71341064b0 2024-05-17T04:54:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f712affd640 1 -- 172.21.15.50:0/4026021712 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7124005020 con 0x7f71341064b0 2024-05-17T04:54:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f712affd640 1 -- 172.21.15.50:0/4026021712 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400ce60 con 0x7f71341064b0 2024-05-17T04:54:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a37c0 con 0x7f71341064b0 2024-05-17T04:54:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.405+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a3cc0 con 0x7f71341064b0 2024-05-17T04:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 Processor -- start 2024-05-17T04:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 -- start start 2024-05-17T04:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f712affd640 1 -- 172.21.15.50:0/4026021712 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7124004a60 con 0x7f71341064b0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f0f41dcc0 con 0x557f0f1081b0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557f0f40d860 con 0x557f0f0e74d0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7142d6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557f0f4118a0 con 0x557f0f0df2f0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59264/0 (socket says 172.21.15.50:59264) 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f713a449640 1 -- 172.21.15.50:0/2703182017 learned_addr learned my addr 172.21.15.50:0/2703182017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7139c48640 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.408+0000 7f7139447640 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 -- 172.21.15.50:0/2703182017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 msgr2=0x557f0f4abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 -- 172.21.15.50:0/2703182017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 msgr2=0x557f0f4b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 -- 172.21.15.50:0/2703182017 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0f3d5d50 con 0x557f0f1081b0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f712affd640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 0x7f71200b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f713a449640 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f712c0098b0 tx=0x7f712c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc062d8ca31b71e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f7139c48640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 0x7f71200b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f712affd640 1 -- 172.21.15.50:0/4026021712 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71240e2560 con 0x7f71341064b0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f711ffff640 1 -- 172.21.15.50:0/2703182017 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c031d00 con 0x557f0f1081b0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f711ffff640 1 -- 172.21.15.50:0/2703182017 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f712c036040 con 0x557f0f1081b0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.409+0000 7f711ffff640 1 -- 172.21.15.50:0/2703182017 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c0305c0 con 0x557f0f1081b0 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.410+0000 7f7142d6d740 1 -- 172.21.15.50:0/2703182017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 msgr2=0x557f0f4ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.410+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f712c0098b0 tx=0x7f712c0098e0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.410+0000 7f7139c48640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 0x7f71200b4cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x557f0f4acb70 tx=0x7f713000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- 172.21.15.50:0/2703182017 shutdown_connections 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2703182017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f4ab770 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- 172.21.15.50:0/2703182017 >> 172.21.15.50:0/2703182017 conn(0x557f0f0fc300 msgr2=0x557f0f4a71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- 172.21.15.50:0/2703182017 shutdown_connections 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- 172.21.15.50:0/2703182017 wait complete. 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- start start 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 Processor -- start 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0df2f0 0x557f0f4b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f1081b0 0x557f0f4bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f0f349ab0 con 0x557f0f0e74d0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557f0f3ca4d0 con 0x557f0f0df2f0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557f0f4b34b0 con 0x557f0f1081b0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.411+0000 7f7142d6d740 10 client.0 _ll_register_callbacks cb 0x557f0f408950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59270/0 (socket says 172.21.15.50:59270) 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139c48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0df2f0 0x557f0f4b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 -- 172.21.15.50:0/2901361148 learned_addr learned my addr 172.21.15.50:0/2901361148 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f713a449640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f1081b0 0x557f0f4bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 -- 172.21.15.50:0/2901361148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f1081b0 msgr2=0x557f0f4bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f1081b0 0x557f0f4bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 -- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0df2f0 msgr2=0x557f0f4b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0df2f0 0x557f0f4b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 -- 172.21.15.50:0/2901361148 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0f3cb340 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7139447640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f712400c6a0 tx=0x7f71240a9680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fd191192fee678 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f711d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712401c030 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mon.0 v2:172.21.15.50:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71240051d0 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f712c036040 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f71240a9ef0 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.412+0000 7f7142d6d740 10 client.16755 fetch_fsmap learned FSMap version 45 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557f0f173790 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mon.0 v2:172.21.15.50:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f712c036040 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mon.0 v2:172.21.15.50:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f71240b21c0 con 0x557f0f0e74d0 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 10 client.16755 fetch_fsmap finished waiting for FSMap version 45 2024-05-17T04:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 20 client.16755 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f0c6d80 0x557f0f4c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 4 client.16755 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f0f41fc10 con 0x557f0f0c6d80 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.413+0000 7f7139c48640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f0c6d80 0x557f0f4c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.414+0000 7f711d7fa640 10 client.16755 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.414+0000 7f7139c48640 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f0c6d80 0x557f0f4c26f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779d02c06bcd1015 server_cookie=cadf95418e050a05 in_seq=0 out_seq=0 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f711d7fa640 1 -- 172.21.15.50:0/2901361148 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f0f41fc10 con 0x557f0f0c6d80 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f711d7fa640 10 client.16755 handle_command_reply: tid=0 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 client.16755 shutdown 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 msgr2=0x557f0f4b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f712400c6a0 tx=0x7f71240a9680 comp rx=0 tx=0).stop 2024-05-17T04:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 shutdown_connections 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f1081b0 0x557f0f4bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f0c6d80 0x557f0f4c26f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4b54b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 --2- 172.21.15.50:0/2901361148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0df2f0 0x557f0f4b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 >> 172.21.15.50:0/2901361148 conn(0x557f0f0fc300 msgr2=0x557f0f4aa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f711d7fa640 0 client.16755 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 shutdown_connections 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 1 -- 172.21.15.50:0/2901361148 wait complete. 2024-05-17T04:54:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.415+0000 7f7142d6d740 20 client.16755 trim_cache size 0 max 16384 2024-05-17T04:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.433+0000 7f7142d6d740 1 Processor -- start 2024-05-17T04:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 -- start start 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f1081b0 0x557f0f4aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71341b09f0 con 0x557f0f0e74d0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ae260 con 0x557f0f1081b0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7142d6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ac0b0 con 0x557f0f0df2f0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59282/0 (socket says 172.21.15.50:59282) 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 -- 172.21.15.50:0/3756924359 learned_addr learned my addr 172.21.15.50:0/3756924359 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139c48640 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f713a449640 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f1081b0 0x557f0f4aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 -- 172.21.15.50:0/3756924359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 msgr2=0x557f0f4afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 -- 172.21.15.50:0/3756924359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f1081b0 msgr2=0x557f0f4aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f1081b0 0x557f0f4aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.434+0000 7f7139447640 1 -- 172.21.15.50:0/3756924359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0f1ec620 con 0x557f0f0e74d0 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.435+0000 7f7139447640 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f71240a6a60 tx=0x7f71240a96f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62538eb42e2f0922 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.435+0000 7f711f7fe640 1 -- 172.21.15.50:0/3756924359 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712401c070 con 0x557f0f0e74d0 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.435+0000 7f711f7fe640 1 -- 172.21.15.50:0/3756924359 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71240adba0 con 0x557f0f0e74d0 2024-05-17T04:54:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.435+0000 7f711f7fe640 1 -- 172.21.15.50:0/3756924359 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240b21c0 con 0x557f0f0e74d0 2024-05-17T04:54:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- 172.21.15.50:0/3756924359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 msgr2=0x557f0f4bc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f71240a6a60 tx=0x7f71240a96f0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- 172.21.15.50:0/3756924359 shutdown_connections 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f1081b0 0x557f0f4aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f0e74d0 0x557f0f4bc3a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 --2- 172.21.15.50:0/3756924359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- 172.21.15.50:0/3756924359 >> 172.21.15.50:0/3756924359 conn(0x557f0f0fc300 msgr2=0x557f0f4a9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- 172.21.15.50:0/3756924359 shutdown_connections 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- 172.21.15.50:0/3756924359 wait complete. 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 -- start start 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.436+0000 7f7142d6d740 1 Processor -- start 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f528a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ac720 con 0x557f0f1081b0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71341adb90 con 0x557f0f0e74d0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71341acdf0 con 0x557f0f0df2f0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7142d6d740 10 client.0 _ll_register_callbacks cb 0x7f71240a5050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f528a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.50:32894/0 (socket says 172.21.15.50:32894) 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 -- 172.21.15.50:0/1454490280 learned_addr learned my addr 172.21.15.50:0/1454490280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f713a449640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f528a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59290/0 (socket says 172.21.15.50:59290) 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139447640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 -- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 msgr2=0x557f0f4bb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4bb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 -- 172.21.15.50:0/1454490280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 msgr2=0x557f0f528a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f528a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139c48640 1 -- 172.21.15.50:0/1454490280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0f4bbd10 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.437+0000 7f7139447640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f7139c48640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7130005d80 tx=0x7f71300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c7ea46a4abec7a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713001e9d0 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f713001eb70 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71240adba0 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f713003b050 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f7142d6d740 10 client.26494 fetch_fsmap learned FSMap version 45 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557f0f52a0b0 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71240adba0 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f71300114f0 con 0x557f0f0df2f0 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.438+0000 7f7142d6d740 10 client.26494 fetch_fsmap finished waiting for FSMap version 45 2024-05-17T04:54:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7142d6d740 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7142d6d740 20 client.26494 populate_metadata read hostname 'smithi050' 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f09c1c0 0x557f0f52ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7142d6d740 4 client.26494 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 --> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f0f4193f0 con 0x557f0f09c1c0 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7139447640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f09c1c0 0x557f0f52ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f711d7fa640 10 client.26494 ms_handle_connect on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.439+0000 7f7139447640 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f09c1c0 0x557f0f52ca80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9d72bfd73ede49 server_cookie=ee81344d0f204c0a in_seq=0 out_seq=0 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f711d7fa640 1 -- 172.21.15.50:0/1454490280 <== mds.0 v2:172.21.15.105:6838/3722840784 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f0f4193f0 con 0x557f0f09c1c0 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f711d7fa640 10 client.26494 handle_command_reply: tid=0 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 client.26494 shutdown 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 msgr2=0x557f0f4baf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7130005d80 tx=0x7f71300046c0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 shutdown_connections 2024-05-17T04:54:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:6838/3722840784,v1:172.21.15.105:6839/3722840784] conn(0x557f0f09c1c0 0x557f0f52ca80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f711d7fa640 0 client.26494 ms_handle_reset on v2:172.21.15.105:6838/3722840784 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f0f1081b0 0x557f0f528a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557f0f0e74d0 0x557f0f4bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 --2- 172.21.15.50:0/1454490280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f0f0df2f0 0x557f0f4baf80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 >> 172.21.15.50:0/1454490280 conn(0x557f0f0fc300 msgr2=0x557f0f4aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 shutdown_connections 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 1 -- 172.21.15.50:0/1454490280 wait complete. 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.443+0000 7f7142d6d740 20 client.26494 trim_cache size 0 max 16384 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:{ 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout: "return_code": 0 2024-05-17T04:54:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stdout:} 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 msgr2=0x7f71200b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 0x7f71200b4cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x557f0f4acb70 tx=0x7f713000b390 comp rx=0 tx=0).stop 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 msgr2=0x7f713419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 0x7f713419bed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9f0 tx=0x7f712400eeb0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 shutdown_connections 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71200b27e0 0x7f71200b4cf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7134106e00 0x7f71341a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71341064b0 0x7f713419bed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 --2- 172.21.15.50:0/4026021712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7134105390 0x7f713419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:54:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 >> 172.21.15.50:0/4026021712 conn(0x7f7134100b70 msgr2=0x7f71341097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:54:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 shutdown_connections 2024-05-17T04:54:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi050.front.sepia.ceph.com.smithi050.stderr:2024-05-17T04:54:53.444+0000 7f713beac640 1 -- 172.21.15.50:0/4026021712 wait complete. 2024-05-17T04:54:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:54 smithi105 ceph-mon[34795]: pgmap v996: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-05-17T04:54:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:54 smithi110 ceph-mon[35497]: pgmap v996: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-05-17T04:54:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:54 smithi050 ceph-mon[31739]: pgmap v996: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-05-17T04:54:55.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.814 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.815 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:55 smithi105 ceph-mon[34795]: pgmap v997: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 934 KiB/s wr, 124 op/s 2024-05-17T04:54:55.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.816 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.817 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:55 smithi110 ceph-mon[35497]: pgmap v997: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 934 KiB/s wr, 124 op/s 2024-05-17T04:54:55.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.825 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:55.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:55 smithi050 ceph-mon[31739]: pgmap v997: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 934 KiB/s wr, 124 op/s 2024-05-17T04:54:57.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:54:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:54:57.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:54:57.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:54:57.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:54:58.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:54:57 smithi105 ceph-mon[34795]: pgmap v998: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-17T04:54:58.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:54:57 smithi110 ceph-mon[35497]: pgmap v998: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-17T04:54:58.075 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:54:57 smithi050 ceph-mon[31739]: pgmap v998: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-17T04:55:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:00 smithi105 ceph-mon[34795]: pgmap v999: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-17T04:55:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:00 smithi110 ceph-mon[35497]: pgmap v999: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-17T04:55:00.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:00 smithi050 ceph-mon[31739]: pgmap v999: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.0 MiB/s wr, 133 op/s 2024-05-17T04:55:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:02 smithi105 ceph-mon[34795]: pgmap v1000: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 986 KiB/s wr, 130 op/s 2024-05-17T04:55:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:02 smithi110 ceph-mon[35497]: pgmap v1000: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 986 KiB/s wr, 130 op/s 2024-05-17T04:55:02.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:02 smithi050 ceph-mon[31739]: pgmap v1000: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 986 KiB/s wr, 130 op/s 2024-05-17T04:55:04.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-17T04:55:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:04 smithi105 ceph-mon[34795]: pgmap v1001: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-17T04:55:04.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:04 smithi110 ceph-mon[35497]: pgmap v1001: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-17T04:55:04.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:04 smithi050 ceph-mon[31739]: pgmap v1001: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/: 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout:total 2 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610575 May 17 04:35 . 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1182610575 May 17 04:35 .. 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 data 2024-05-17T04:55:04.722 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data: 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:total 46 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 . 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610575 May 17 04:35 .. 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 base 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 17 04:36 current_logfiles 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 17 04:36 global 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 17 04:36 log 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_commit_ts 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_dynshmem 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 17 04:35 pg_hba.conf 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 17 04:35 pg_ident.conf 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:55 pg_logical 2024-05-17T04:55:04.723 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 pg_multixact 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_notify 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_replslot 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_serial 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_snapshots 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 17 04:55 pg_stat 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:55 pg_stat_tmp 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 May 17 04:48 pg_subtrans 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_tblspc 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 pg_twophase 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:35 PG_VERSION 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 637534208 May 17 04:55 pg_wal 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 pg_xact 2024-05-17T04:55:04.724 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 17 04:35 postgresql.auto.conf 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 17 04:35 postgresql.conf 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 17 04:36 postmaster.opts 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base: 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:total 3 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 . 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 1 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 13493 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 528273967 May 17 04:43 13494 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:38 pgsql_tmp 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-17T04:55:04.725 INFO:teuthology.orchestra.run.smithi050.stdout:total 7778 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 . 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 .. 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 112 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 113 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 1247 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1247_vm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:36 1249 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1249_fsm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1249_vm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:55:04.726 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:36 1259 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1259_vm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13346 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13348 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13351 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13353 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:55:04.727 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13356 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13358 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 13361 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 13363 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 1417 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 1418 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 174 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 175 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2187 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2224 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2328 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2336 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2337 2024-05-17T04:55:04.728 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2579 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:55:04.729 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2604 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606_vm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:36 2608 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2608_vm 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:55:04.730 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2610 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2610_vm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2611 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2613 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:55:04.731 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 04:36 2619 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2619_fsm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2619_vm 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2620 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2650 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2651 2024-05-17T04:55:04.732 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2652 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2653 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2654 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2655 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2656 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2657 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:36 2658 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 2659 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2660 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2661 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2662 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2663 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2664 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2665 2024-05-17T04:55:04.733 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2666 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2667 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2668 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2669 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2670 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:36 2673 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:36 2674 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2678 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2679 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2680 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:55:04.734 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2683 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2686 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2687 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:35 2688 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:35 2689 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2696 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2699 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2701 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2702 2024-05-17T04:55:04.735 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2703 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2704 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2754 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2755 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2756 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 2757 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2830 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2831 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2832 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2833 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2834 2024-05-17T04:55:04.736 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2835 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2836 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2837 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2995 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2996 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:55:04.737 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3118 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3119 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3256 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3257 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3258 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3350 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3351 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3379 2024-05-17T04:55:04.738 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3380 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3381 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3429 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3430 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3431 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3433 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3439 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3440 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3455 2024-05-17T04:55:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3466 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3467 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3468 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3501 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3502 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3503 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3534 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3542 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3574 2024-05-17T04:55:04.740 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3575 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3576 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3596 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3597 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3598 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3599 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:55:04.741 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3606 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 3607 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:55:04.742 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3997 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4143 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4144 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4145 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4146 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4147 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4148 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4149 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4150 2024-05-17T04:55:04.743 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4151 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4152 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4153 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4154 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4155 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4156 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4157 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4158 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4159 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4160 2024-05-17T04:55:05.009 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4161 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4162 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4163 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4164 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4165 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4166 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4167 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4168 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4169 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4170 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4171 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4172 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4173 2024-05-17T04:55:05.010 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4174 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 5002 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 548 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 549 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6102 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6104 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6106 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6110 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6111 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6112 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6113 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6117 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 826 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 827 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 828 2024-05-17T04:55:05.011 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:35 pg_filenode.map 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:35 PG_VERSION 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:total 7778 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 17 04:36 . 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 .. 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 112 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 113 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 1247 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1247_vm 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:36 1249 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1249_fsm 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1249_vm 2024-05-17T04:55:05.012 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:36 1259 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1259_vm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13346 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13348 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13351 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13353 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:55:05.013 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13356 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13358 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13361 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13363 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1417 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1418 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 174 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 175 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2187 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2224 2024-05-17T04:55:05.014 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2328 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2336 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2337 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2579 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:55:05.015 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2604 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2606_vm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:36 2608 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2608_vm 2024-05-17T04:55:05.016 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2610 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2610_vm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2611 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2613 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:55:05.017 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 17 04:36 2619 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2619_fsm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2619_vm 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2620 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2650 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2651 2024-05-17T04:55:05.018 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2652 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2653 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2654 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2655 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2656 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2657 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:36 2658 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:36 2659 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2660 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2661 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2662 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2663 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2664 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2665 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2666 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2667 2024-05-17T04:55:05.019 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2668 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2669 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2670 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:36 2673 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:36 2674 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2678 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2679 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2680 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2683 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:55:05.020 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2686 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2687 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2688 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2689 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2696 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2699 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2701 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2702 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2703 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2704 2024-05-17T04:55:05.021 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2754 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2755 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2756 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2757 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2830 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2831 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2832 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2833 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2834 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2835 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2836 2024-05-17T04:55:05.022 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2837 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2995 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2996 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:55:05.023 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3118 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3119 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3256 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3257 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3258 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3350 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3351 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3379 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3380 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3381 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:55:05.024 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3429 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3430 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3431 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3433 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3439 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3440 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3455 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3466 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3467 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3468 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3501 2024-05-17T04:55:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3502 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3503 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3534 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3542 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3574 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3575 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3576 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3596 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3597 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3598 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3599 2024-05-17T04:55:05.026 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3606 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3607 2024-05-17T04:55:05.027 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3997 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4143 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4144 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4145 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4146 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4147 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4148 2024-05-17T04:55:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4149 2024-05-17T04:55:05.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4150 2024-05-17T04:55:05.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4151 2024-05-17T04:55:05.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4152 2024-05-17T04:55:05.029 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4153 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4154 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4155 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4156 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4157 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4158 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4159 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4160 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4161 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4162 2024-05-17T04:55:05.327 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4163 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4164 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4165 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4166 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4167 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4168 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4169 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4170 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4171 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4172 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4173 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4174 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 5002 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 548 2024-05-17T04:55:05.328 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 549 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6102 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6104 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6106 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6110 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6111 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6112 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6113 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6117 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 826 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 827 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 828 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:36 pg_filenode.map 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:36 PG_VERSION 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.329 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:total 515895 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 528273967 May 17 04:43 . 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 .. 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 112 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 113 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:40 1247 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1247_fsm 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 1247_vm 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 17 04:41 1249 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:39 1249_fsm 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 1249_vm 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 17 04:36 1255 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1255_fsm 2024-05-17T04:55:05.330 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1255_vm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 17 04:54 1259 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1259_fsm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 1259_vm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 17 04:36 13344 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13344_fsm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13344_vm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13346 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13348 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13349_fsm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13349_vm 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13351 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13353 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354 2024-05-17T04:55:05.331 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13354_fsm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13354_vm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13356 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13358 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 13359_fsm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13359_vm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 13361 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 13363 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1417 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 1418 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 438173696 May 17 04:55 16396 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 17 04:54 16396_fsm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:54 16396_vm 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1736704 May 17 04:54 16397 2024-05-17T04:55:05.332 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:54 16397_fsm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:54 16397_vm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 794624 May 17 04:54 16399 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:54 16399_fsm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:54 16399_vm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:48 16400 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:48 16402 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 17 04:55 16404 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 7192576 May 17 04:55 16406 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:54 16406_fsm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:54 16406_vm 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 174 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 175 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2187 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2224 2024-05-17T04:55:05.333 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2328 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2336 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2337 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:40 2579 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2600 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2600_fsm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2600_vm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2601_fsm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2601_vm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2602 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2602_fsm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2602_vm 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2603 2024-05-17T04:55:05.334 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2603_fsm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2603_vm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2604 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2605 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2605_fsm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2605_vm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:40 2606 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2606_fsm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 2606_vm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2607 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2607_fsm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2607_vm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 17 04:40 2608 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2608_fsm 2024-05-17T04:55:05.335 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 2608_vm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 17 04:36 2609 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2609_fsm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2609_vm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:41 2610 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2610_fsm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 2610_vm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2611 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2612_fsm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2612_vm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2613 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2615_fsm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2615_vm 2024-05-17T04:55:05.336 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2616 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2616_fsm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2616_vm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 17 04:36 2617 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2617_fsm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2617_vm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 17 04:36 2618 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2618_fsm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2618_vm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 17 04:54 2619 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:47 2619_fsm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:39 2619_vm 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2620 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2650 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2651 2024-05-17T04:55:05.337 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2652 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2653 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2654 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2655 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2656 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2657 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 17 04:40 2658 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 17 04:41 2659 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2660 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2661 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:41 2662 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:41 2663 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:40 2664 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:40 2665 2024-05-17T04:55:05.338 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:40 2666 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:40 2667 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2668 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2669 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2670 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 17 04:40 2673 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 17 04:40 2674 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 17 04:36 2675 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:41 2678 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:41 2679 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2680 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2681 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2682 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2683 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2684 2024-05-17T04:55:05.339 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2685 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2686 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2687 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 2688 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 17 04:36 2689 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 17 04:36 2690 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 17 04:36 2691 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2692 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2693 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:51 2696 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2699 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2701 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2702 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:40 2703 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:40 2704 2024-05-17T04:55:05.340 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2753 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2753_fsm 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2753_vm 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2754 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2755 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2756 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 2757 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2830 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2831 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2832 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2833 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2834 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2835 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2836 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2837 2024-05-17T04:55:05.341 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 17 04:36 2838 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2838_fsm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2838_vm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2839 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2840_fsm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2840_vm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2841 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 2995 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2996 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3079_fsm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3079_vm 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3080 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3081 2024-05-17T04:55:05.342 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:36 3085 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3118 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3119 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 17 04:36 3164 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3256 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3257 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3258 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3350 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3351 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3379 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3380 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3381 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3394 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3394_fsm 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3394_vm 2024-05-17T04:55:05.343 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3395 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3429 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3430 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3431 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3433 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3439 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3440 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 17 04:40 3455 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 17 04:36 3456 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3456_fsm 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3456_vm 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3466 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3467 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3468 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3501 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3502 2024-05-17T04:55:05.344 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3503 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3534 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3541_fsm 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3541_vm 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3542 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3574 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3575 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3576 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3596 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3597 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 3598 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3599 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3600_fsm 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3600_vm 2024-05-17T04:55:05.345 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3601_fsm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3601_vm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3602_fsm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3602_vm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3603_fsm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3603_vm 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3604 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3605 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3606 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3607 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3608 2024-05-17T04:55:05.346 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:36 3609 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3712 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 3764_fsm 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3764_vm 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3766 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 3767 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 3997 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4143 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4144 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4145 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4146 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4147 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4148 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4149 2024-05-17T04:55:05.347 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4150 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4151 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4152 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4153 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4154 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4155 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4156 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4157 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4158 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4159 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4160 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4161 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4162 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4163 2024-05-17T04:55:05.348 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4164 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4165 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4166 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4167 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4168 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4169 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4170 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4171 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4172 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 4173 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 4174 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 5002 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 548 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 549 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6102 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6104 2024-05-17T04:55:05.349 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 6106 2024-05-17T04:55:05.350 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6110 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6111 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6112 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6113 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 6117 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:36 826 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 827 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 828 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:36 pg_filenode.map 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 17 04:36 pg_internal.init 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 17 04:36 PG_VERSION 2024-05-17T04:55:05.375 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:38 . 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 544200245 May 17 04:38 .. 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/global: 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:total 561 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 17 04:36 . 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1213 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1213_fsm 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1213_vm 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1214 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1214_fsm 2024-05-17T04:55:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1214_vm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 1232 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 1233 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1260 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1260_fsm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1260_vm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1261 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1261_fsm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1261_vm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1262 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 1262_fsm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 1262_vm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2396 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 17 04:36 2396_fsm 2024-05-17T04:55:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:36 2396_vm 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2397 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2671 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2672 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2676 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2677 2024-05-17T04:55:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2694 2024-05-17T04:55:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:36 2695 2024-05-17T04:55:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2697 2024-05-17T04:55:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 2698 2024-05-17T04:55:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2846 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2847 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2964 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2965 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 2966 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 2967 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 3592 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 3593 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4060 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4061 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4175 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4176 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4177 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4178 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4181 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4182 2024-05-17T04:55:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4183 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4184 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 4185 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 4186 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6000 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6001 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6002 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 6100 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6114 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 6115 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:55 pg_control 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 17 04:35 pg_filenode.map 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 17 04:36 pg_internal.init 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/log: 2024-05-17T04:55:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:total 3 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 17 04:36 . 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 17 04:55 postgresql-Fri.log 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-17T04:55:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:total 3 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:55 . 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 mappings 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:55 replorigin_checkpoint 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 snapshots 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.432 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:55 .. 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 17 04:55 .. 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:total 2 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 . 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 members 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 offsets 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-17T04:55:05.433 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 .. 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 0000 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:total 9 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:35 . 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 17 04:35 .. 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 17 04:41 0000 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.434 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.435 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:total 16 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 17 04:55 . 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 17 04:55 db_0.stat 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 17 04:55 db_13494.stat 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 17 04:55 global.stat 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:55 . 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:total 201 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 May 17 04:48 . 2024-05-17T04:55:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 May 17 04:54 0001 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.470 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:total 622594 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 637534208 May 17 04:55 . 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:55 000000010000000000000044 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:41 000000010000000000000045 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:41 000000010000000000000046 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:42 000000010000000000000047 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:45 000000010000000000000048 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:42 000000010000000000000049 2024-05-17T04:55:05.471 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:42 00000001000000000000004A 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:41 00000001000000000000004B 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:42 00000001000000000000004C 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:43 00000001000000000000004D 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:45 00000001000000000000004E 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:44 00000001000000000000004F 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:41 000000010000000000000050 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:41 000000010000000000000051 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:47 000000010000000000000052 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:46 000000010000000000000053 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:50 000000010000000000000054 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:49 000000010000000000000055 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:47 000000010000000000000056 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:50 000000010000000000000057 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:48 000000010000000000000058 2024-05-17T04:55:05.472 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:46 000000010000000000000059 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:48 00000001000000000000005A 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:47 00000001000000000000005B 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:46 00000001000000000000005C 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:48 00000001000000000000005D 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:50 00000001000000000000005E 2024-05-17T04:55:05.473 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:49 00000001000000000000005F 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:47 000000010000000000000060 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:46 000000010000000000000061 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:51 000000010000000000000062 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:52 000000010000000000000063 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:53 000000010000000000000064 2024-05-17T04:55:05.477 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:53 000000010000000000000065 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:52 000000010000000000000066 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:51 000000010000000000000067 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:52 000000010000000000000068 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 17 04:51 000000010000000000000069 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 archive_status 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:total 1 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 17 04:35 . 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 637534208 May 17 04:55 .. 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:total 33 2024-05-17T04:55:05.478 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:35 . 2024-05-17T04:55:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1182610468 May 17 04:55 .. 2024-05-17T04:55:05.479 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 17 04:54 0000 2024-05-17T04:55:05.480 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-17T04:55:05.490 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:55:05.498 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-17T04:55:05.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T04:55:05.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.671+0000 7f79b9724640 1 -- 172.21.15.50:0/2330604219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4104270 msgr2=0x7f79b4104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:05.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.671+0000 7f79b9724640 1 --2- 172.21.15.50:0/2330604219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4104270 0x7f79b4104680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009900 tx=0x7f79a802a410 comp rx=0 tx=0).stop 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- 172.21.15.50:0/2330604219 shutdown_connections 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- 172.21.15.50:0/2330604219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b410b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- 172.21.15.50:0/2330604219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4104270 0x7f79b4104680 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- 172.21.15.50:0/2330604219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4103150 0x7f79b4103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- 172.21.15.50:0/2330604219 >> 172.21.15.50:0/2330604219 conn(0x7f79b40fe950 msgr2=0x7f79b4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- 172.21.15.50:0/2330604219 shutdown_connections 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- 172.21.15.50:0/2330604219 wait complete. 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 Processor -- start 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- start start 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 0x7f79b4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410eb30 con 0x7f79b4103150 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.672+0000 7f79b9724640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410e9b0 con 0x7f79b4104bc0 2024-05-17T04:55:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b9724640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410ecb0 con 0x7f79b4104270 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55894/0 (socket says 172.21.15.50:55894) 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 -- 172.21.15.50:0/3162210361 learned_addr learned my addr 172.21.15.50:0/3162210361 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b27fc640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 0x7f79b4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b37fe640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 -- 172.21.15.50:0/3162210361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 msgr2=0x7f79b4199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 0x7f79b4199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 -- 172.21.15.50:0/3162210361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 msgr2=0x7f79b41a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b41a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 -- 172.21.15.50:0/3162210361 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41a1270 con 0x7f79b4103150 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b27fc640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 0x7f79b4199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b37fe640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b41a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b2ffd640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4017070 con 0x7f79b4103150 2024-05-17T04:55:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41a1560 con 0x7f79b4103150 2024-05-17T04:55:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a4009c30 con 0x7f79b4103150 2024-05-17T04:55:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a400f620 con 0x7f79b4103150 2024-05-17T04:55:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.673+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a1ab0 con 0x7f79b4103150 2024-05-17T04:55:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.674+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b410a410 con 0x7f79b4103150 2024-05-17T04:55:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.676+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f79a401a070 con 0x7f79b4103150 2024-05-17T04:55:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.677+0000 7f799bfff640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 0x7f799c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.677+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f79a40e2180 con 0x7f79b4103150 2024-05-17T04:55:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.677+0000 7f79b27fc640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 0x7f799c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.677+0000 7f79b27fc640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 0x7f799c0b4d00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006010 tx=0x7f79a8031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.680+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79a4009de0 con 0x7f79b4103150 2024-05-17T04:55:05.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.873+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79b4063940 con 0x7f79b4103150 2024-05-17T04:55:05.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.875+0000 7f799bfff640 1 -- 172.21.15.50:0/3162210361 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7f79a40a9b10 con 0x7f79b4103150 2024-05-17T04:55:05.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:05.876 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:55:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 msgr2=0x7f799c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 0x7f799c0b4d00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006010 tx=0x7f79a8031420 comp rx=0 tx=0).stop 2024-05-17T04:55:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 msgr2=0x7f79b41997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).stop 2024-05-17T04:55:05.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 shutdown_connections 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79b4104bc0 0x7f79b41a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79b4104270 0x7f79b4199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f799c0b27f0 0x7f799c0b4d00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 --2- 172.21.15.50:0/3162210361 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79b4103150 0x7f79b41997e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 >> 172.21.15.50:0/3162210361 conn(0x7f79b40fe950 msgr2=0x7f79b41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 shutdown_connections 2024-05-17T04:55:05.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:05.877+0000 7f79b9724640 1 -- 172.21.15.50:0/3162210361 wait complete. 2024-05-17T04:55:05.891 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T04:26:42.688129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T04:31:37.906052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T04:31:41.858212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-17T04:31:53.969356+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-17T04:55:05.892 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T04:55:06.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.008+0000 7fd4b596f640 1 -- 172.21.15.50:0/4037975653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0101a90 msgr2=0x7fd4b010b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.008+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4037975653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0101a90 0x7fd4b010b6b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009930 tx=0x7fd4a802a420 comp rx=0 tx=0).stop 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 -- 172.21.15.50:0/4037975653 shutdown_connections 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4037975653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0101a90 0x7fd4b010b6b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4037975653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 0x7fd4b0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4037975653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0104700 0x7fd4b0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 -- 172.21.15.50:0/4037975653 >> 172.21.15.50:0/4037975653 conn(0x7fd4b00fcfe0 msgr2=0x7fd4b00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 -- 172.21.15.50:0/4037975653 shutdown_connections 2024-05-17T04:55:06.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 -- 172.21.15.50:0/4037975653 wait complete. 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 Processor -- start 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.009+0000 7fd4b596f640 1 -- start start 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 0x7fd4b01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 0x7fd4b0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010eb40 con 0x7fd4b0104700 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010e9c0 con 0x7fd4b0101a90 2024-05-17T04:55:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4b596f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010ecc0 con 0x7fd4b0101140 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55900/0 (socket says 172.21.15.50:55900) 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 -- 172.21.15.50:0/4271572925 learned_addr learned my addr 172.21.15.50:0/4271572925 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4ae7fc640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 0x7fd4b0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4aeffd640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 0x7fd4b01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 -- 172.21.15.50:0/4271572925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 msgr2=0x7fd4b01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 0x7fd4b01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 -- 172.21.15.50:0/4271572925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 msgr2=0x7fd4b0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 0x7fd4b0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 -- 172.21.15.50:0/4271572925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01a1320 con 0x7fd4b0104700 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4ae7fc640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 0x7fd4b0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.010+0000 7fd4af7fe640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ab0 tx=0x7fd4a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.011+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803f030 con 0x7fd4b0104700 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.011+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd4a8032e10 con 0x7fd4b0104700 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.011+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b01a15b0 con 0x7fd4b0104700 2024-05-17T04:55:06.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.011+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8035070 con 0x7fd4b0104700 2024-05-17T04:55:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.011+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b01a1aa0 con 0x7fd4b0104700 2024-05-17T04:55:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.012+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b010a530 con 0x7fd4b0104700 2024-05-17T04:55:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.013+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd4a8042030 con 0x7fd4b0104700 2024-05-17T04:55:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.014+0000 7fd4b496d640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 0x7fd4980b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.014+0000 7fd4aeffd640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 0x7fd4980b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.014+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd4a8102a20 con 0x7fd4b0104700 2024-05-17T04:55:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.014+0000 7fd4aeffd640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 0x7fd4980b4d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006b020 tx=0x7fd4a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.018+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4a80c9d60 con 0x7fd4b0104700 2024-05-17T04:55:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.214+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4b0063940 con 0x7fd4b0104700 2024-05-17T04:55:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.214+0000 7fd4b496d640 1 -- 172.21.15.50:0/4271572925 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fd4a80ce060 con 0x7fd4b0104700 2024-05-17T04:55:06.215 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 1 2024-05-17T04:55:06.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 msgr2=0x7fd4980b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 0x7fd4980b4d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd4b006b020 tx=0x7fd4a0009500 comp rx=0 tx=0).stop 2024-05-17T04:55:06.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 msgr2=0x7fd4b01a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ab0 tx=0x7fd4a80042f0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 shutdown_connections 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd4980b27f0 0x7fd4980b4d00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4b0104700 0x7fd4b01a0cb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4b0101a90 0x7fd4b0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 --2- 172.21.15.50:0/4271572925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4b0101140 0x7fd4b01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 >> 172.21.15.50:0/4271572925 conn(0x7fd4b00fcfe0 msgr2=0x7fd4b0107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 shutdown_connections 2024-05-17T04:55:06.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.216+0000 7fd4b596f640 1 -- 172.21.15.50:0/4271572925 wait complete. 2024-05-17T04:55:06.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 -- 172.21.15.50:0/3769472861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc103ab0 msgr2=0x7f71fc10a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 --2- 172.21.15.50:0/3769472861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc103ab0 0x7f71fc10a3b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009ca0 tx=0x7f71f402a8c0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 -- 172.21.15.50:0/3769472861 shutdown_connections 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 --2- 172.21.15.50:0/3769472861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc103ab0 0x7f71fc10a3b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 --2- 172.21.15.50:0/3769472861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 --2- 172.21.15.50:0/3769472861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc102040 0x7f71fc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 -- 172.21.15.50:0/3769472861 >> 172.21.15.50:0/3769472861 conn(0x7f71fc0fd860 msgr2=0x7f71fc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 -- 172.21.15.50:0/3769472861 shutdown_connections 2024-05-17T04:55:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.347+0000 7f720214b640 1 -- 172.21.15.50:0/3769472861 wait complete. 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 Processor -- start 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 -- start start 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 0x7f71fc1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10d9d0 con 0x7f71fc102040 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10d850 con 0x7f71fc103ab0 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f720214b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc10db50 con 0x7f71fc103160 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55912/0 (socket says 172.21.15.50:55912) 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 -- 172.21.15.50:0/2855466391 learned_addr learned my addr 172.21.15.50:0/2855466391 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fbfff640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 0x7f71fc1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71faffd640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 -- 172.21.15.50:0/2855466391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 msgr2=0x7f71fc19e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc19e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 -- 172.21.15.50:0/2855466391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 msgr2=0x7f71fc1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.348+0000 7f71fb7fe640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 0x7f71fc1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71fb7fe640 1 -- 172.21.15.50:0/2855466391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc1a12a0 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71faffd640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71fbfff640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 0x7f71fc1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71fb7fe640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00e9c0 tx=0x7f71ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00cd80 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc1a1590 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71ec0054b0 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec010430 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.349+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc1a1ae0 con 0x7f71fc102040 2024-05-17T04:55:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.350+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc109290 con 0x7f71fc102040 2024-05-17T04:55:06.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.352+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f71ec004f20 con 0x7f71fc102040 2024-05-17T04:55:06.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.353+0000 7f71f8ff9640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 0x7f71e40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.353+0000 7f71faffd640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 0x7f71e40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.353+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71ec0e2f40 con 0x7f71fc102040 2024-05-17T04:55:06.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.353+0000 7f71faffd640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 0x7f71e40b4d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71fc06a440 tx=0x7f71f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.356+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71ec0aa8d0 con 0x7f71fc102040 2024-05-17T04:55:06.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.549+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71fc063940 con 0x7f71fc102040 2024-05-17T04:55:06.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.550+0000 7f71f8ff9640 1 -- 172.21.15.50:0/2855466391 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f71ec0aa1d0 con 0x7f71fc102040 2024-05-17T04:55:06.551 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-05-17T04:55:06.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 msgr2=0x7f71e40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 0x7f71e40b4d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71fc06a440 tx=0x7f71f0005f90 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 msgr2=0x7f71fc06aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00e9c0 tx=0x7f71ec00ee80 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 shutdown_connections 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71fc103ab0 0x7f71fc1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71fc103160 0x7f71fc19e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f71e40b27f0 0x7f71e40b4d00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 --2- 172.21.15.50:0/2855466391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71fc102040 0x7f71fc06aaa0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 >> 172.21.15.50:0/2855466391 conn(0x7f71fc0fd860 msgr2=0x7f71fc106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 shutdown_connections 2024-05-17T04:55:06.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.552+0000 7f720214b640 1 -- 172.21.15.50:0/2855466391 wait complete. 2024-05-17T04:55:06.564 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:06 smithi050 ceph-mon[31739]: pgmap v1002: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-17T04:55:06.564 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:06 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3162210361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:55:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:06 smithi105 ceph-mon[34795]: pgmap v1002: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-17T04:55:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:06 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3162210361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:55:06.565 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T04:55:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:06 smithi110 ceph-mon[35497]: pgmap v1002: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-17T04:55:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:06 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3162210361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 -- 172.21.15.50:0/659798655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c103160 msgr2=0x7fbb0c103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 --2- 172.21.15.50:0/659798655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c103160 0x7fbb0c103570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009900 tx=0x7fbb0002a410 comp rx=0 tx=0).stop 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 -- 172.21.15.50:0/659798655 shutdown_connections 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 --2- 172.21.15.50:0/659798655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 0x7fbb0c10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 --2- 172.21.15.50:0/659798655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c103160 0x7fbb0c103570 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 --2- 172.21.15.50:0/659798655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c102040 0x7fbb0c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 -- 172.21.15.50:0/659798655 >> 172.21.15.50:0/659798655 conn(0x7fbb0c0fd860 msgr2=0x7fbb0c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 -- 172.21.15.50:0/659798655 shutdown_connections 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.689+0000 7fbb11aba640 1 -- 172.21.15.50:0/659798655 wait complete. 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 Processor -- start 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 -- start start 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 0x7fbb0c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 0x7fbb0c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10da20 con 0x7fbb0c102040 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10d8a0 con 0x7fbb0c103ab0 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb11aba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10dba0 con 0x7fbb0c103160 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55926/0 (socket says 172.21.15.50:55926) 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb0b7fe640 1 -- 172.21.15.50:0/1683442353 learned_addr learned my addr 172.21.15.50:0/1683442353 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb0bfff640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 0x7fbb0c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.690+0000 7fbb0affd640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 0x7fbb0c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 -- 172.21.15.50:0/1683442353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 msgr2=0x7fbb0c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 0x7fbb0c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 -- 172.21.15.50:0/1683442353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 msgr2=0x7fbb0c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 0x7fbb0c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 -- 172.21.15.50:0/1683442353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c1a1230 con 0x7fbb0c102040 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0affd640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 0x7fbb0c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb0b7fe640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c970 tx=0x7fbafc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc014070 con 0x7fbb0c102040 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbafc004440 con 0x7fbb0c102040 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c1a1520 con 0x7fbb0c102040 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc003d40 con 0x7fbb0c102040 2024-05-17T04:55:06.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.691+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a1a70 con 0x7fbb0c102040 2024-05-17T04:55:06.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.692+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c109290 con 0x7fbb0c102040 2024-05-17T04:55:06.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.694+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbafc017070 con 0x7fbb0c102040 2024-05-17T04:55:06.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.694+0000 7fbb08ff9640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 0x7fbaf40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:06.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.694+0000 7fbb0affd640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 0x7fbaf40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:06.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.695+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbafc0e2470 con 0x7fbb0c102040 2024-05-17T04:55:06.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.695+0000 7fbb0affd640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 0x7fbaf40b4d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005f70 tx=0x7fbb00005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:06.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.699+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbafc01e020 con 0x7fbb0c102040 2024-05-17T04:55:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.890+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb0c063940 con 0x7fbb0c102040 2024-05-17T04:55:06.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.890+0000 7fbb08ff9640 1 -- 172.21.15.50:0/1683442353 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fbafc0a3e20 con 0x7fbb0c102040 2024-05-17T04:55:06.891 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 64 2024-05-17T04:55:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 msgr2=0x7fbaf40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 0x7fbaf40b4d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005f70 tx=0x7fbb00005720 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 msgr2=0x7fbb0c06aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c970 tx=0x7fbafc00ce30 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 shutdown_connections 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb0c103ab0 0x7fbb0c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb0c103160 0x7fbb0c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fbaf40b27f0 0x7fbaf40b4d00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 --2- 172.21.15.50:0/1683442353 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb0c102040 0x7fbb0c06aaa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.892+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 >> 172.21.15.50:0/1683442353 conn(0x7fbb0c0fd860 msgr2=0x7fbb0c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.893+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 shutdown_connections 2024-05-17T04:55:06.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:06.893+0000 7fbb11aba640 1 -- 172.21.15.50:0/1683442353 wait complete. 2024-05-17T04:55:06.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-17T04:55:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 -- 172.21.15.50:0/712471543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164103150 msgr2=0x7fb164103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/712471543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164103150 0x7fb164103560 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 -- 172.21.15.50:0/712471543 shutdown_connections 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/712471543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104bc0 0x7fb16410b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/712471543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164104270 0x7fb164104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/712471543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164103150 0x7fb164103560 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 -- 172.21.15.50:0/712471543 >> 172.21.15.50:0/712471543 conn(0x7fb1640fe950 msgr2=0x7fb164100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 -- 172.21.15.50:0/712471543 shutdown_connections 2024-05-17T04:55:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.022+0000 7fb16c0fd640 1 -- 172.21.15.50:0/712471543 wait complete. 2024-05-17T04:55:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 Processor -- start 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 -- start start 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164103150 0x7fb1641997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 0x7fb164199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410eb30 con 0x7fb164104bc0 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410e9b0 con 0x7fb164103150 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16c0fd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410ecb0 con 0x7fb164104270 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16a69a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16a69a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55952/0 (socket says 172.21.15.50:55952) 2024-05-17T04:55:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.023+0000 7fb16a69a640 1 -- 172.21.15.50:0/2726607463 learned_addr learned my addr 172.21.15.50:0/2726607463 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb169e99640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164103150 0x7fb1641997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb169698640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 0x7fb164199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 -- 172.21.15.50:0/2726607463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 msgr2=0x7fb164199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 0x7fb164199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 -- 172.21.15.50:0/2726607463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164103150 msgr2=0x7fb1641997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164103150 0x7fb1641997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 -- 172.21.15.50:0/2726607463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1641a12f0 con 0x7fb164104bc0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb169698640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 0x7fb164199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16a69a640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00eaa0 tx=0x7fb15c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cdb0 con 0x7fb164104bc0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb15c005020 con 0x7fb164104bc0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1641a15e0 con 0x7fb164104bc0 2024-05-17T04:55:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0116d0 con 0x7fb164104bc0 2024-05-17T04:55:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.024+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641a1b30 con 0x7fb164104bc0 2024-05-17T04:55:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.025+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16410a3a0 con 0x7fb164104bc0 2024-05-17T04:55:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.027+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb15c0051d0 con 0x7fb164104bc0 2024-05-17T04:55:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.027+0000 7fb15affd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 0x7fb1500b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.027+0000 7fb169e99640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 0x7fb1500b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.027+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb15c0e2b20 con 0x7fb164104bc0 2024-05-17T04:55:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.028+0000 7fb169e99640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 0x7fb1500b4d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb160005ab0 tx=0x7fb160000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:07.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.030+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb15c016030 con 0x7fb164104bc0 2024-05-17T04:55:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.223+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb16402de30 con 0x7fb164104bc0 2024-05-17T04:55:07.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.224+0000 7fb15affd640 1 -- 172.21.15.50:0/2726607463 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fb15c0aa4b0 con 0x7fb164104bc0 2024-05-17T04:55:07.225 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 32 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 msgr2=0x7fb1500b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 0x7fb1500b4d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb160005ab0 tx=0x7fb160000bc0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 msgr2=0x7fb1641a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00eaa0 tx=0x7fb15c00ef60 comp rx=0 tx=0).stop 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 shutdown_connections 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb1500b27f0 0x7fb1500b4d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb164104bc0 0x7fb1641a0c80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb164104270 0x7fb164199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 --2- 172.21.15.50:0/2726607463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb164103150 0x7fb1641997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 >> 172.21.15.50:0/2726607463 conn(0x7fb1640fe950 msgr2=0x7fb164107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:07.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 shutdown_connections 2024-05-17T04:55:07.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.226+0000 7fb16c0fd640 1 -- 172.21.15.50:0/2726607463 wait complete. 2024-05-17T04:55:07.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-17T04:55:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 -- 172.21.15.50:0/3247309809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103160 msgr2=0x7f1560103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 --2- 172.21.15.50:0/3247309809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103160 0x7f1560103570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1548009900 tx=0x7f154802a410 comp rx=0 tx=0).stop 2024-05-17T04:55:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 -- 172.21.15.50:0/3247309809 shutdown_connections 2024-05-17T04:55:07.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 --2- 172.21.15.50:0/3247309809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560103ab0 0x7f156010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 --2- 172.21.15.50:0/3247309809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103160 0x7f1560103570 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 --2- 172.21.15.50:0/3247309809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560102040 0x7f1560102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 -- 172.21.15.50:0/3247309809 >> 172.21.15.50:0/3247309809 conn(0x7f15600fd860 msgr2=0x7f15600ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 -- 172.21.15.50:0/3247309809 shutdown_connections 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.353+0000 7f1565e22640 1 -- 172.21.15.50:0/3247309809 wait complete. 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 Processor -- start 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 -- start start 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560103160 0x7f156019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f156010da80 con 0x7f1560103ab0 2024-05-17T04:55:07.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f156010d900 con 0x7f1560102040 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f1565e22640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f156010dc00 con 0x7f1560103160 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155ffff640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55976/0 (socket says 172.21.15.50:55976) 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.50:41820/0 (socket says 172.21.15.50:41820) 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155ffff640 1 -- 172.21.15.50:0/397023058 learned_addr learned my addr 172.21.15.50:0/397023058 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.354+0000 7f155effd640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560103160 0x7f156019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 -- 172.21.15.50:0/397023058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560103160 msgr2=0x7f156019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560103160 0x7f156019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 -- 172.21.15.50:0/397023058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 msgr2=0x7f156006aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:07.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 -- 172.21.15.50:0/397023058 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15601a1370 con 0x7f1560103ab0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155f7fe640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155ffff640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f155800d750 tx=0x7f155800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558017070 con 0x7f1560103ab0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15580054b0 con 0x7f1560103ab0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601a1660 con 0x7f1560103ab0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558004b90 con 0x7f1560103ab0 2024-05-17T04:55:07.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.355+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601a1be0 con 0x7f1560103ab0 2024-05-17T04:55:07.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.356+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15601092a0 con 0x7f1560103ab0 2024-05-17T04:55:07.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.358+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1558004d30 con 0x7f1560103ab0 2024-05-17T04:55:07.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.359+0000 7f155cff9640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 0x7f154c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:07.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.359+0000 7f155f7fe640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 0x7f154c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.359+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f15580e2440 con 0x7f1560103ab0 2024-05-17T04:55:07.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.359+0000 7f155f7fe640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 0x7f154c0b4d00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1560103060 tx=0x7f155000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:07.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.362+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15580a9dd0 con 0x7f1560103ab0 2024-05-17T04:55:07.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:07 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/4271572925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:07 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2855466391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:07 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1683442353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:07 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/4271572925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:07 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2855466391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:07 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1683442353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:07 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/4271572925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:07 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2855466391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:07 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1683442353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-17T04:55:07.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.607+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1560063940 con 0x7f1560103ab0 2024-05-17T04:55:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.616+0000 7f155cff9640 1 -- 172.21.15.50:0/397023058 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815642 (secure 0 0 0) 0x7f15580a96d0 con 0x7f1560103ab0 2024-05-17T04:55:07.627 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:55:07.627 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 18, 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "active_gid": 14152, 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "active_name": "x", 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addrs": { 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6800", 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1662041432 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.628 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6801", 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1662041432 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addr": "172.21.15.50:6801/1662041432", 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "active_change": "2024-05-17T04:24:18.593502+0000", 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "active_mgr_features": 4540701547738038271, 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "standbys": [ 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.629 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 24109, 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "z", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.630 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.631 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.632 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.633 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.634 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.635 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-17T04:55:07.636 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-17T04:55:07.637 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-17T04:55:07.638 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-17T04:55:07.639 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.640 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.641 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.643 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-17T04:55:07.644 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.645 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-17T04:55:07.646 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-17T04:55:07.647 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.650 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T04:55:07.652 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.655 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.658 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.659 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.660 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-17T04:55:07.661 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.662 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.663 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.664 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.665 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.666 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-17T04:55:07.667 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-17T04:55:07.668 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.669 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.670 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-17T04:55:07.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T04:55:07.672 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-17T04:55:07.673 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.674 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.675 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.677 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.678 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.680 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.681 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-17T04:55:07.682 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.683 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.684 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.686 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.687 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.688 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.689 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.690 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.691 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.692 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-17T04:55:07.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-17T04:55:07.694 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T04:55:07.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.697 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.698 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.699 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.700 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-17T04:55:07.701 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.702 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T04:55:07.703 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.704 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.705 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.706 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T04:55:07.707 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.708 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.709 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.710 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.711 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.712 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T04:55:07.713 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.714 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.715 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.716 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.717 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.718 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.719 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-17T04:55:07.720 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-17T04:55:07.721 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.722 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.723 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.724 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.725 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.727 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-17T04:55:07.728 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-17T04:55:07.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-17T04:55:07.730 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.731 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-17T04:55:07.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.733 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.738 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.739 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.740 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.741 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.742 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.743 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.744 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.745 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.746 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.747 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.748 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.749 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.750 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.751 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.752 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.753 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.754 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.757 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T04:55:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T04:55:07.759 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T04:55:07.760 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T04:55:07.761 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.762 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.763 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.764 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.765 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.766 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T04:55:07.767 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.768 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T04:55:07.769 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T04:55:07.770 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T04:55:07.771 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.772 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.773 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.774 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.775 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.776 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.777 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.778 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.779 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.780 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.781 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-17T04:55:07.782 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.783 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.784 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.785 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.786 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.787 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.788 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.790 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.791 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.792 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.793 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.794 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.795 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.797 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.798 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.799 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.800 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.801 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.802 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.804 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.805 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.806 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.807 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.809 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.810 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.811 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.812 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.813 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.814 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.815 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.816 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.817 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-17T04:55:07.818 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-17T04:55:07.819 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-17T04:55:07.820 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-17T04:55:07.821 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.822 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.823 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.824 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.825 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.826 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.827 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.828 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.829 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.830 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.831 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.832 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.833 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.834 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:07.835 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.836 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.837 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.838 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.840 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.841 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.842 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.843 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.844 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.845 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.846 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.847 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-17T04:55:07.848 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.849 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.850 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.851 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.852 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.853 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.854 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.855 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.856 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.857 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.858 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.859 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.860 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.861 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.862 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.863 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.864 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.865 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.866 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.867 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.868 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.869 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.870 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.871 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.872 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.874 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.875 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.876 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-17T04:55:07.877 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-17T04:55:07.878 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.879 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.880 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.881 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.882 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.883 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.884 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.885 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.886 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.887 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.888 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.889 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-17T04:55:07.890 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.891 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.892 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.893 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.894 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.895 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.896 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.897 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.898 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.899 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.900 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.901 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.902 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.903 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-17T04:55:07.904 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.905 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.906 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.907 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.908 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.909 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.910 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.911 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T04:55:07.912 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.913 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.914 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.915 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.916 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.917 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.918 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.919 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.920 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.921 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.922 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.923 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.924 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.925 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.926 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.927 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.928 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.929 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.931 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.932 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.933 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.934 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.935 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.936 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.937 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.938 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.939 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.940 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.941 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-17T04:55:07.942 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.943 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.944 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.945 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.946 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.947 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.948 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.949 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.950 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.951 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.952 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.953 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.954 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.955 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.956 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.957 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.958 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.959 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.960 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.961 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.962 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-17T04:55:07.963 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.964 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.965 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.966 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.967 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.968 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.969 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:07.970 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.971 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.973 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.974 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.975 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.976 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.977 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.978 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.979 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-17T04:55:07.980 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:07.982 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.983 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.984 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.985 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.986 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.987 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.988 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.989 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.990 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.992 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.993 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.995 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:07.996 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.997 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:07.998 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:07.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.000 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.001 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.002 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.003 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.004 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.005 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-17T04:55:08.006 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.007 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.008 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.009 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.010 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-17T04:55:08.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-17T04:55:08.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-17T04:55:08.014 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-17T04:55:08.015 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.016 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.017 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.018 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.019 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.020 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.021 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.022 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.024 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.025 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.026 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.027 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.028 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.029 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.030 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.031 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.032 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.033 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.034 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.036 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.037 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.038 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.040 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.041 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.042 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.043 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.044 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.045 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.046 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.048 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.049 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.050 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.051 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.052 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.053 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.054 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-17T04:55:08.055 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.057 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.058 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.059 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.060 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.061 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.062 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.063 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.064 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.065 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.066 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.068 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.069 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.070 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.071 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-17T04:55:08.074 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.075 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.076 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.078 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.080 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.081 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.082 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.084 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.085 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.086 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.087 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.088 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.089 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.090 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.092 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.093 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.094 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.095 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.096 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.097 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.098 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.099 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.100 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.101 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.102 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.103 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-17T04:55:08.104 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.105 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.106 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.107 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.108 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.109 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.110 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.112 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-17T04:55:08.113 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-17T04:55:08.114 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.115 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "gid": 24116, 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "y", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "mgr_features": 4540701547738038271, 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.116 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.117 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.118 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.119 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.120 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.121 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-17T04:55:08.123 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.124 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.125 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.126 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.127 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.128 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.129 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.130 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.131 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.132 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.133 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.134 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.135 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.136 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-17T04:55:08.137 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.138 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.139 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.140 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.141 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.142 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.143 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.147 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.147 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.147 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.148 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.149 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.150 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-17T04:55:08.151 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.152 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.153 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.154 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.155 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.156 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.157 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.158 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.159 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.160 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.161 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.162 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.163 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-17T04:55:08.164 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.165 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.166 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.167 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T04:55:08.168 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.170 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.171 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.172 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.173 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.174 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.175 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.176 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.177 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.178 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.179 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-17T04:55:08.180 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-17T04:55:08.181 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-17T04:55:08.182 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-17T04:55:08.183 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.184 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-17T04:55:08.185 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.186 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-17T04:55:08.187 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-17T04:55:08.188 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-17T04:55:08.189 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-17T04:55:08.190 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-17T04:55:08.191 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-17T04:55:08.192 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-17T04:55:08.193 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.194 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.195 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.196 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.197 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.198 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.199 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.200 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.201 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.202 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.203 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.204 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T04:55:08.205 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.206 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.207 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.208 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.209 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-17T04:55:08.210 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.211 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.213 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.214 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.215 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.216 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-17T04:55:08.217 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.218 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.219 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T04:55:08.220 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.221 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.222 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.223 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.224 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.225 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.226 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.227 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.228 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.229 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.230 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-17T04:55:08.231 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.232 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.233 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.234 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.235 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.236 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.237 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.238 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.239 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.240 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.241 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.242 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.244 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.245 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.246 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.247 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.248 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.249 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.250 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.251 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.252 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.253 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.254 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.256 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.257 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.258 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.259 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.260 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.262 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.265 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.266 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.267 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.268 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.269 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-17T04:55:08.270 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.271 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.272 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.274 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.275 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.276 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.277 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.278 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.279 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.280 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.281 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.283 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.285 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.286 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.287 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.288 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.289 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.290 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.291 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.292 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.293 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.294 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.295 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.296 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.297 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:08.298 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.299 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.300 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.301 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.302 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.303 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.304 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.305 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.306 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.308 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.309 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.311 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.312 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.313 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.314 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.315 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.316 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.317 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.318 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.320 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-17T04:55:08.322 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.323 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-17T04:55:08.325 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-17T04:55:08.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-17T04:55:08.327 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.328 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.329 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.330 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.331 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.332 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.333 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.334 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.335 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.336 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:08.337 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-17T04:55:08.338 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.339 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.340 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-17T04:55:08.341 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.342 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.343 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.344 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.345 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.346 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.348 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.349 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.350 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.351 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.352 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-17T04:55:08.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-17T04:55:08.355 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.356 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.357 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.358 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.359 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.360 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.361 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.362 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.363 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.364 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.365 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.366 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.367 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.368 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-17T04:55:08.369 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.370 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.371 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.372 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.373 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.374 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.376 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.377 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.378 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.379 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.380 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.381 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.382 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.383 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.386 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.387 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.388 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.389 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.390 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.391 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.392 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.393 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-17T04:55:08.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.396 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.397 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.398 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.399 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-17T04:55:08.400 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.401 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.402 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.403 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.404 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.406 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.407 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.408 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.409 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.410 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.411 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.412 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.413 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.414 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.415 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.416 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.417 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.418 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.419 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.420 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.421 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.422 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.423 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.424 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.425 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.426 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.427 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.428 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.430 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.431 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.432 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.433 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.434 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.435 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-17T04:55:08.436 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-17T04:55:08.437 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.438 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.439 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.440 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.441 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.442 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-17T04:55:08.443 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.444 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.445 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.446 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.447 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.448 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.449 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.450 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.451 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.452 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.453 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.454 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.455 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.456 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.457 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.458 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.459 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.460 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-17T04:55:08.461 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.462 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.463 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.464 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.466 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.467 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.468 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.469 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.470 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.471 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.472 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.473 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.474 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.475 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.476 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.477 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.478 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.479 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.480 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.481 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.482 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.483 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.484 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.485 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.486 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.487 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.489 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.490 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.491 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.492 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.493 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.494 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.495 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.496 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.498 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.499 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-17T04:55:08.500 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.501 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.502 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.503 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.504 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.505 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.506 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.507 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.508 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.509 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.510 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.511 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.512 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.513 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.514 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-17T04:55:08.515 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.516 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.517 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.518 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.519 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.520 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.522 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.523 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.524 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.525 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.526 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.527 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.528 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.529 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.530 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.531 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.532 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.533 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.534 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.535 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.536 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.537 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.538 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.539 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.540 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.541 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.542 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.543 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.544 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-17T04:55:08.545 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-17T04:55:08.546 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-17T04:55:08.547 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.548 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.549 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-17T04:55:08.550 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-17T04:55:08.551 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:08.552 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.553 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.554 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.555 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.556 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.558 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.559 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.560 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.561 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.562 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.563 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.564 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.565 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T04:55:08.566 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.567 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.568 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.569 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.570 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.571 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.573 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.574 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.575 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.576 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.577 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.578 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-17T04:55:08.579 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.580 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.581 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.582 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.583 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.585 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.586 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.587 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.588 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.589 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.590 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.591 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.592 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.593 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.594 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.595 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.596 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard", 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "nfs", 2024-05-17T04:55:08.597 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.598 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.599 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.600 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.601 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.602 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.603 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.604 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.605 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2024-05-17T04:55:08.606 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2024-05-17T04:55:08.607 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2024-05-17T04:55:08.608 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2024-05-17T04:55:08.609 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2024-05-17T04:55:08.610 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2024-05-17T04:55:08.611 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.612 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.613 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.614 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.615 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:08.616 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2024-05-17T04:55:08.617 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.618 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.619 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.620 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2359", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.621 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "6", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.622 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.623 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.624 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.625 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.626 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2024-05-17T04:55:08.627 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "upmap", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2024-05-17T04:55:08.629 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "read", 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap", 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap-read" 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2024-05-17T04:55:08.630 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.632 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.633 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.634 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_optimizations": { 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_optimizations", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.635 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.636 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm", 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_down_multiplier": { 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_down_multiplier", 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.637 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_refresh_rate": { 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_refresh_rate", 2024-05-17T04:55:08.638 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "20", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.639 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "agent_starting_port": { 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "agent_starting_port", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4721", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.640 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_ptrace": { 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_ptrace", 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.641 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_interval": { 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_interval", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.642 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.643 INFO:teuthology.orchestra.run.smithi050.stdout: "autotune_memory_target_ratio": { 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "autotune_memory_target_ratio", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.7", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.644 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm_log_destination": { 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cephadm_log_destination", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.645 INFO:teuthology.orchestra.run.smithi050.stdout: "file", 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "file,syslog", 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "syslog" 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.646 INFO:teuthology.orchestra.run.smithi050.stdout: "cgroups_split": { 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cgroups_split", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.647 INFO:teuthology.orchestra.run.smithi050.stdout: "config_checks_enabled": { 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_checks_enabled", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.648 INFO:teuthology.orchestra.run.smithi050.stdout: "config_dashboard": { 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_dashboard", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.649 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_alertmanager": { 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_alertmanager", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.650 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_base": { 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_base", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Container image name, without the tag", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.651 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_elasticsearch": { 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_elasticsearch", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "elasticsearch container image", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.652 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_grafana": { 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_grafana", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_haproxy": { 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_haproxy", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "HAproxy container image", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.654 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_agent": { 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_agent", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger agent container image", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_collector": { 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_collector", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger collector container image", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.656 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_jaeger_query": { 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_jaeger_query", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Jaeger query container image", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.657 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_keepalived": { 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_keepalived", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Keepalived container image", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_loki": { 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_loki", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Loki container image", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_node_exporter": { 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_node_exporter", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.660 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_nvmeof": { 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_nvmeof", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Nvme-of container image", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.661 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_prometheus": { 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_prometheus", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Prometheus container image", 2024-05-17T04:55:08.662 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_promtail": { 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_promtail", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.663 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Promtail container image", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_samba": { 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_samba", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-17T04:55:08.664 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Samba/SMB container image", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "container_image_snmp_gateway": { 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_image_snmp_gateway", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.665 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SNMP Gateway container image", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "container_init": { 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "container_init", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.666 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Run podman/docker with `--init`", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "daemon_cache_timeout": { 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "daemon_cache_timeout", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.667 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "default_cephadm_command_timeout": { 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_cephadm_command_timeout", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.668 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "900", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.669 INFO:teuthology.orchestra.run.smithi050.stdout: "default_registry": { 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "default_registry", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "docker.io", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-17T04:55:08.670 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "device_cache_timeout": { 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_cache_timeout", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1800", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.671 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache device inventory", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "device_enhanced_scan": { 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_enhanced_scan", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.672 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "facts_cache_timeout": { 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "facts_cache_timeout", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.673 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "seconds to cache host facts data", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.674 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "grafana_dashboards_path": { 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "grafana_dashboards_path", 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.675 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "host_check_interval": { 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "host_check_interval", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.676 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to perform a host check", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "hw_monitoring": { 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hw_monitoring", 2024-05-17T04:55:08.677 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.678 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_list_all": { 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_list_all", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.679 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.681 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "log_refresh_metadata": { 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_refresh_metadata", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.682 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.683 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.684 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.685 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.686 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "*", 2024-05-17T04:55:08.687 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max_count_per_host": { 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_count_per_host", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.688 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of daemons per service per host", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "max_osd_draining_count": { 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_osd_draining_count", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.689 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "migration_current": { 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "migration_current", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.690 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "internal - do not modify", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.691 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "root", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm-package", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "root" 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "mode for remote execution of cephadm", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "oob_default_addr": { 2024-05-17T04:55:08.692 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "oob_default_addr", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "169.254.1.1", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "prometheus_alerts_path": { 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus_alerts_path", 2024-05-17T04:55:08.693 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_insecure": { 2024-05-17T04:55:08.694 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_insecure", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_password": { 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_password", 2024-05-17T04:55:08.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_url": { 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_url", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.696 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "registry_username": { 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "registry_username", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.697 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "secure_monitoring_stack": { 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "secure_monitoring_stack", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.698 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "service_discovery_port": { 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "service_discovery_port", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8765", 2024-05-17T04:55:08.699 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "cephadm service discovery port", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.700 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.701 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "use_agent": { 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_agent", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.702 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "use_repo_digest": { 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "use_repo_digest", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.703 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_failed_host_check": { 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_failed_host_check", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.704 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if the host check fails", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_daemons": { 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_daemons", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.705 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_on_stray_hosts": { 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_on_stray_hosts", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.706 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.707 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.708 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.709 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.710 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.711 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.712 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2024-05-17T04:55:08.713 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.714 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.715 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.716 INFO:teuthology.orchestra.run.smithi050.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.717 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.719 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.720 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.721 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.722 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-17T04:55:08.723 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-17T04:55:08.724 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-17T04:55:08.725 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-17T04:55:08.726 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-17T04:55:08.727 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-17T04:55:08.728 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.729 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.730 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.731 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2024-05-17T04:55:08.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-17T04:55:08.733 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-17T04:55:08.734 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.735 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-17T04:55:08.736 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.737 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.738 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "[]", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.739 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "MULTICLUSTER_CONFIG": { 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "{}", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.740 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.741 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.742 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.743 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.744 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.745 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.746 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.747 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.748 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.749 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_ENABLED": { 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.750 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.751 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.752 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8", 2024-05-17T04:55:08.753 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.754 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.756 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.757 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.758 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "UNSAFE_TLS_v1_2": { 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.759 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.761 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.762 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "cross_origin_url": { 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cross_origin_url", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.763 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.764 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.765 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.766 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.767 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.768 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.769 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.770 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.771 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.772 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "motd": { 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "motd", 2024-05-17T04:55:08.773 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "The message of the day", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect_resolve_ip_addr": { 2024-05-17T04:55:08.774 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "redirect_resolve_ip_addr", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.775 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.776 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.777 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.778 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.779 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.780 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:08.781 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2024-05-17T04:55:08.782 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.783 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.784 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.785 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.787 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.788 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.789 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2024-05-17T04:55:08.790 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2024-05-17T04:55:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.792 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.793 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.794 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.795 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.796 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.797 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.798 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.799 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.800 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.801 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.802 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.803 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.804 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "predictor_model": { 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predictor_model", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.805 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "prophetstor", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.806 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.807 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.808 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-17T04:55:08.809 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-17T04:55:08.810 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server hostname", 2024-05-17T04:55:08.811 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "5", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-17T04:55:08.812 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.813 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.814 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.815 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.816 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.817 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "password of InfluxDB server user", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.818 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "InfluxDB server port", 2024-05-17T04:55:08.819 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.820 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.821 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "32", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-17T04:55:08.822 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "username of InfluxDB server user", 2024-05-17T04:55:08.823 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-17T04:55:08.824 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.825 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.826 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.827 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.828 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.829 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.830 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.831 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.832 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.833 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.834 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.835 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.836 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.837 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "k8sevents", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_event_retention_days": { 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ceph_event_retention_days", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.838 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "config_check_secs": { 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "config_check_secs", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "10", 2024-05-17T04:55:08.839 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.840 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.841 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.842 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.843 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.844 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.845 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.846 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.847 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.848 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.849 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.850 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.851 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.852 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.853 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.854 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2024-05-17T04:55:08.855 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2024-05-17T04:55:08.856 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.857 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2024-05-17T04:55:08.858 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.859 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mds_autoscaler", 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.861 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.862 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.863 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.864 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.865 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.866 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirroring", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.867 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.868 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.869 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.870 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.871 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.872 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "nfs", 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.873 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.874 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.875 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.876 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.877 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.878 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "fail_fs": { 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "fail_fs", 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.880 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.881 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.882 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.883 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.884 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.885 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.886 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "cephadm", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "rook", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "test_orchestrator" 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Orchestrator backend", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.887 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.888 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_perf_query", 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.889 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.890 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.891 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.892 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.893 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.894 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.895 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "osd_support", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.896 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.897 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.898 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.899 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.900 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.901 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.902 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.903 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.904 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.905 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.906 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.907 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.908 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "threshold": { 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threshold", 2024-05-17T04:55:08.909 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3.0", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1.0", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "scaling threshold", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.910 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_pg_recovery_event": { 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_pg_recovery_event", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.911 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.912 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.913 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.914 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.915 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.916 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.917 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2024-05-17T04:55:08.918 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2024-05-17T04:55:08.919 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long the module is going to sleep", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.920 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.921 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "cache": { 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "cache", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.922 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "exclude_perf_counters": { 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "exclude_perf_counters", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-17T04:55:08.923 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.924 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.925 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.926 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.927 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.928 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.929 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "300", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.930 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15.0", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.931 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.932 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "9283", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.933 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.934 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "stale_cache_strategy": { 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stale_cache_strategy", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "log", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.935 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "default", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.936 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "default", 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "error" 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2024-05-17T04:55:08.937 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.938 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.939 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.940 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.941 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.942 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.943 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_snap_create": { 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_snap_create", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.944 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "mirror_snapshot_schedule": { 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mirror_snapshot_schedule", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.945 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.946 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "trash_purge_schedule": { 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "trash_purge_schedule", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.947 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.948 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_auth": { 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_auth", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.949 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.950 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.951 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.952 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.953 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.954 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.955 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.956 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.957 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.958 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rgw", 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.959 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.960 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.961 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.962 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.963 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.964 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.965 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.967 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.968 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.970 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.971 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "storage_class": { 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "storage_class", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "local", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-17T04:55:08.972 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.973 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.974 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.975 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.976 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.977 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.978 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.979 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.980 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.981 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.982 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.983 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.984 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.985 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.986 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption7": { 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption7", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.987 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "42", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.988 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.989 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2024-05-17T04:55:08.990 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2024-05-17T04:55:08.991 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.992 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smb", 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.994 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.995 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:08.996 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:08.997 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:08.998 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:08.999 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "update_orchestration": { 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "update_orchestration", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.000 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snap_schedule", 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "allow_m_granularity": { 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "allow_m_granularity", 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.001 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "allow minute scheduled snapshots", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "dump_on_update": { 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dump_on_update", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.002 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "dump database to debug log on update", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.003 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.004 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.005 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.006 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.007 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.008 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "stats", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.010 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.013 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.014 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.015 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2024-05-17T04:55:09.016 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.017 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.018 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.019 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.020 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.021 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.022 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2024-05-17T04:55:09.023 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:09.024 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.025 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.026 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.027 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.028 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.029 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.030 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.031 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.032 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2024-05-17T04:55:09.033 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2024-05-17T04:55:09.034 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2024-05-17T04:55:09.035 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.036 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_perf": { 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_perf", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.037 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.038 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.039 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.040 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.041 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.042 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.043 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.044 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard_description": { 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard_description", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.045 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.046 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.048 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.049 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.050 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2024-05-17T04:55:09.051 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2024-05-17T04:55:09.052 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.053 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.054 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.055 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.056 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.057 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.058 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.059 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.060 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.061 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-17T04:55:09.062 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.063 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.064 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.065 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.066 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.067 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max_concurrent_clones": { 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_concurrent_clones", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "4", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-17T04:55:09.068 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "periodic_async_work": { 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "periodic_async_work", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.069 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Periodically check for async work", 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_delay": { 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_delay", 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.070 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "snapshot_clone_no_wait": { 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "snapshot_clone_no_wait", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.071 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.072 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2024-05-17T04:55:09.073 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "discovery_interval": { 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "discovery_interval", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "100", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.074 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.075 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "log_level": { 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_level", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.077 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster": { 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.078 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_cluster_level": { 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_cluster_level", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "info", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.079 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "critical", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "debug", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "info", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "warning" 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "log_to_file": { 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "log_to_file", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2024-05-17T04:55:09.080 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "sqlite3_killpoint": { 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sqlite3_killpoint", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.081 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "dev", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2024-05-17T04:55:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2024-05-17T04:55:09.083 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2024-05-17T04:55:09.084 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "services": { 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dashboard": "https://172.21.15.50:8443/" 2024-05-17T04:55:09.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "always_on_modules": { 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "octopus": [ 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-17T04:55:09.086 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "pacific": [ 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "quincy": [ 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-17T04:55:09.087 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "reef": [ 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-17T04:55:09.088 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "squid": [ 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_autoscaler", 2024-05-17T04:55:09.089 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "telemetry", 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "last_failure_osd_epoch": 3, 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "active_clients": [ 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-17T04:55:09.090 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1804966223 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "libcephsqlite", 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-17T04:55:09.091 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 869091459 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1594728217 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:09.092 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:0", 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 1063028425 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-17T04:55:09.093 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:55:09.094 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:08 smithi110 ceph-mon[35497]: pgmap v1003: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-05-17T04:55:09.094 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:08 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2726607463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T04:55:09.094 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:08 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/397023058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T04:55:09.095 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:08 smithi105 ceph-mon[34795]: pgmap v1003: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-05-17T04:55:09.095 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:08 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2726607463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T04:55:09.095 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:08 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/397023058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 msgr2=0x7f154c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 0x7f154c0b4d00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1560103060 tx=0x7f155000d040 comp rx=0 tx=0).stop 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 msgr2=0x7f15601a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f155800d750 tx=0x7f155800dc10 comp rx=0 tx=0).stop 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 shutdown_connections 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f154c0b27f0 0x7f154c0b4d00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1560103ab0 0x7f15601a0b50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:09.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1560103160 0x7f156019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:09.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 --2- 172.21.15.50:0/397023058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1560102040 0x7f156006aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:09.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 >> 172.21.15.50:0/397023058 conn(0x7f15600fd860 msgr2=0x7f1560106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:09.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 shutdown_connections 2024-05-17T04:55:09.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:07.622+0000 7f1565e22640 1 -- 172.21.15.50:0/397023058 wait complete. 2024-05-17T04:55:09.103 DEBUG:tasks.check_counter:Getting stats from a 2024-05-17T04:55:09.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-17T04:55:09.104 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:08 smithi050 ceph-mon[31739]: pgmap v1003: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-05-17T04:55:09.104 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:08 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2726607463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-17T04:55:09.104 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:08 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/397023058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-17T04:55:09.370 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:10.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:10 smithi050 ceph-mon[31739]: pgmap v1004: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-17T04:55:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:10 smithi105 ceph-mon[34795]: pgmap v1004: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-17T04:55:10.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:10 smithi110 ceph-mon[35497]: pgmap v1004: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-05-17T04:55:12.335 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:55:12.335 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:12.335 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-17T04:55:12.335 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-17T04:55:12.335 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001535698, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000249825, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:12.336 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 380, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 377, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 559488, 2024-05-17T04:55:12.337 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 233856, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.047241312, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022227016, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.030026453, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 376, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.014628441, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000038905 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.338 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 559488, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 233856 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 6, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 292, 2024-05-17T04:55:12.339 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3264, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 209120, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.020108603, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.015020588, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.002446867, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 291, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.015302805, 2024-05-17T04:55:12.340 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000052586 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 209120 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-17T04:55:12.341 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:12.342 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:12.343 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-17T04:55:12.344 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148704, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 29, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:12.345 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:12.346 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 382, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 505746, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 382, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 382, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 505746, 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-17T04:55:12.347 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:12.348 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:12.348 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:12.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:12.348 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:12.348 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:55:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:12 smithi105 ceph-mon[34795]: pgmap v1005: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-17T04:55:12.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:12 smithi110 ceph-mon[35497]: pgmap v1005: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-17T04:55:12.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:12 smithi050 ceph-mon[31739]: pgmap v1005: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-17T04:55:13.464 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-17T04:55:13.464 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-17T04:55:13.465 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-17T04:55:13.465 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-17T04:55:13.465 DEBUG:tasks.check_counter:Getting stats from d 2024-05-17T04:55:13.465 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-17T04:55:13.736 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:14 smithi105 ceph-mon[34795]: pgmap v1006: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-05-17T04:55:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:14 smithi110 ceph-mon[35497]: pgmap v1006: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-05-17T04:55:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:14 smithi050 ceph-mon[31739]: pgmap v1006: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-05-17T04:55:16.006 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001447213, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000342873, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 6, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 297, 2024-05-17T04:55:16.008 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3264, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 210000, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.020305706, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.015552356, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.002033758, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 296, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.011486666, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000038806 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:16.009 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 210000 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 388, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 385, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 562352, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 238192, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.049490615, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022889836, 2024-05-17T04:55:16.010 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.031813334, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 384, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.015961520, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000041566 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 562352, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 238192 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.011 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:16.012 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:16.013 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148721, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 30, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-17T04:55:16.014 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:16.015 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 390, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 507487, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 390, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 390, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 507487, 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-17T04:55:16.016 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:16.017 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:16.017 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:16.017 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:16.017 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:16.017 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:55:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:16 smithi105 ceph-mon[34795]: pgmap v1007: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-17T04:55:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:16 smithi110 ceph-mon[35497]: pgmap v1007: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-17T04:55:16.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:16 smithi050 ceph-mon[31739]: pgmap v1007: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-17T04:55:16.766 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-17T04:55:16.766 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-17T04:55:16.766 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-17T04:55:16.766 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-17T04:55:16.766 DEBUG:tasks.check_counter:Getting stats from g 2024-05-17T04:55:16.767 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-17T04:55:17.020 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:18 smithi105 ceph-mon[34795]: pgmap v1008: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-17T04:55:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:18 smithi110 ceph-mon[35497]: pgmap v1008: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-17T04:55:18.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:18 smithi050 ceph-mon[31739]: pgmap v1008: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 516 KiB/s wr, 62 op/s 2024-05-17T04:55:19.056 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 4, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 2944, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 448, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001831608, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000043599, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000680103, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:19.057 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000778279, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000259426 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.058 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 292, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 209024, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.020478312, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.015787132, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.001885096, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 290, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.010927744, 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000037681 2024-05-17T04:55:19.059 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 209024 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 384, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 381, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 560640, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 235936, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:19.060 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.046816355, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022304470, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.029606090, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 380, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.014852269, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000039084 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 560640, 2024-05-17T04:55:19.061 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 235936 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:19.062 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:19.063 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:19.064 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148761, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 31, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:19.065 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 386, 2024-05-17T04:55:19.066 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 506330, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 386, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 386, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 506330, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:19.067 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:55:19.812 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-17T04:55:19.813 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-17T04:55:19.813 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-17T04:55:19.813 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-17T04:55:19.813 DEBUG:tasks.check_counter:Getting stats from j 2024-05-17T04:55:19.813 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-17T04:55:20.069 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:20.331 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:20 smithi050 ceph-mon[31739]: pgmap v1009: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 150 KiB/s wr, 24 op/s 2024-05-17T04:55:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:20 smithi105 ceph-mon[34795]: pgmap v1009: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 150 KiB/s wr, 24 op/s 2024-05-17T04:55:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:20 smithi110 ceph-mon[35497]: pgmap v1009: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 150 KiB/s wr, 24 op/s 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 0, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 0, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 0, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.001316408, 2024-05-17T04:55:21.803 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.000271166, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:21.804 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 395, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 390, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 566112, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 240560, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.048603432, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.022794330, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.031480998, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:21.805 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 390, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.016262352, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000041698 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 566112, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 240560 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.806 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 297, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 320, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 209904, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.021136732, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.016393249, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.001811922, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 295, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.012781964, 2024-05-17T04:55:21.807 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000043328 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 209904 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 0, 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-05-17T04:55:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:21.809 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:21.810 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 148721, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 30, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:21.811 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 2, 2024-05-17T04:55:21.812 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 393, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 507877, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 393, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-05-17T04:55:21.813 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 393, 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 507877, 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-05-17T04:55:21.814 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-17T04:55:22.501 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:22 smithi050 ceph-mon[31739]: pgmap v1010: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-05-17T04:55:22.502 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-17T04:55:22.502 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-17T04:55:22.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-17T04:55:22.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-17T04:55:22.502 DEBUG:tasks.check_counter:Getting stats from b 2024-05-17T04:55:22.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-17T04:55:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:22 smithi105 ceph-mon[34795]: pgmap v1010: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-05-17T04:55:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:22 smithi110 ceph-mon[35497]: pgmap v1010: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-05-17T04:55:22.820 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:55:24.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:24 smithi105 ceph-mon[34795]: pgmap v1011: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:24.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:24 smithi110 ceph-mon[35497]: pgmap v1011: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:24 smithi050 ceph-mon[31739]: pgmap v1011: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:24.939 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-05-17T04:55:24.939 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:24.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 767956, 2024-05-17T04:55:24.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 667231, 2024-05-17T04:55:24.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 194470695, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 172351141, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 182, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 30.281239399, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 12.029842152, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 22.641264204, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.071735840, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 667232, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 10.164482275, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000015233 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:24.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1413293, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.193500014, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000136 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 508131, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 761830, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 168363309, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 301376118, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 14, 2024-05-17T04:55:24.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 26.143934202, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 11.290080477, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 16.385510472, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.211037279, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 761826, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 11.683750903, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000015336 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1237355, 2024-05-17T04:55:24.942 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.179717512, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000145 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 57088, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 632880 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 302617, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 158954, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 82618052, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 38763581, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 10.157761351, 2024-05-17T04:55:24.943 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 3.946474313, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 8.087001832, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.070369413, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 158953, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.678198388, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000023140 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 455077, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.053284461, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000117 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.944 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 692896, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 245360 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6997, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.828847258, 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000547212 2024-05-17T04:55:24.945 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 674, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.030554920, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000045333 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 113535, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 112314, 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-05-17T04:55:24.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 112314, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 140.861064235, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001254171 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 1221, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 1158, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 3, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 826, 2024-05-17T04:55:24.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 816, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 10, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 826, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 901, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 806, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 806, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 9, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 221706, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 219619, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 1186, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 4, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 2, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 16131, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 49, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 888, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 42, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 706, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 999, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1182610702, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-05-17T04:55:24.949 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 12, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 330237, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 310662, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:24.950 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 390787, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 1773, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 49, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 1746 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 92, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 9, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 6, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 7, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 289, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:24.951 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 154, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 153, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 1, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 51, 2024-05-17T04:55:24.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 3, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 6928, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 6858, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 6858, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 70, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 1, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 94, 2024-05-17T04:55:24.953 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 93, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 93, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 1, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 1, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 1, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 21828374, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 22018990, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2771, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 13.701954967, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004944769 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.954 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 821, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1829, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1008, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 65, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 79, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 14, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 826, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1900, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 1074, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 806, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1821, 2024-05-17T04:55:24.955 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 1015, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 143956, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 231824 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 208198, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 1157, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 113535, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 57, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 2862, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1101, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 6.533559011, 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005934204 2024-05-17T04:55:24.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 104708, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 94.372913283, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000901296 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002064904, 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002064904 2024-05-17T04:55:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3504, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 36.841111844, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.010514015 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.958 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.959 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.960 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 37, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.045927119, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001241273 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-05-17T04:55:24.961 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2685, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.770016613, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000659224 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 104, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.148409442, 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001427013 2024-05-17T04:55:24.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 143, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.069938981, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.007482090 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.963 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.009443469, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004721734 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:24.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001464122, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000732061 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.965 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-05-17T04:55:24.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022811585, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002851448 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 19, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.043403862, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002284413 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:24.967 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 3, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-05-17T04:55:24.968 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 4325, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 4325, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1172, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 360, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 360, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:24.969 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:24.970 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:24.971 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 291739, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 119, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 170, 2024-05-17T04:55:24.972 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 2745845, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 14055, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-05-17T04:55:24.973 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 7773, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 20990430, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 37, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 7773, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7773, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 229.235513207, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.029491253 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7773, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 13325 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.974 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 7773, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 86, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 7687, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 1089, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 2123, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 86, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 2857, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 360, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 3, 2024-05-17T04:55:24.975 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 853, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 3214, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-05-17T04:55:24.976 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-05-17T04:55:24.977 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 89, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 2595, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 145 2024-05-17T04:55:24.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 18, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-05-17T04:55:24.979 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 8, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 333, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 153, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1578700, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 303336670, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1578700, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:24.980 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1578700, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 303336670, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:24.981 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 7773, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 21007091, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 7773, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 21007091, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:24.982 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 7773, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 7773, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 7773, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 7773, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 153, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 16661, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 153, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 86, 2024-05-17T04:55:24.984 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 16661, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x56012fc8e820": { 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 6928, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 17824686, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:24.985 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 6928, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2771, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 17824686, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:24.986 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-05-17T04:55:25.708 INFO:tasks.check_counter:Daemon mds.b mds.exported=49 2024-05-17T04:55:25.708 INFO:tasks.check_counter:Daemon mds.b mds.imported=42 2024-05-17T04:55:25.708 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=92 2024-05-17T04:55:25.708 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9 2024-05-17T04:55:25.708 DEBUG:tasks.check_counter:Getting stats from e 2024-05-17T04:55:25.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-17T04:55:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:55:26.428 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:26 smithi105 ceph-mon[34795]: pgmap v1012: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-05-17T04:55:26.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:26 smithi110 ceph-mon[35497]: pgmap v1012: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-05-17T04:55:26.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:26 smithi050 ceph-mon[31739]: pgmap v1012: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-05-17T04:55:27.866 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-05-17T04:55:27.866 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:27.866 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1229, 2024-05-17T04:55:27.866 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1231, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 21285917, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 390886, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.098759369, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.027508811, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.035838810, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.027412534, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1229, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.029653074, 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000024127 2024-05-17T04:55:27.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1192, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1485, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 21271160, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 957487, 2024-05-17T04:55:27.868 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.109961528, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.032574125, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.037400889, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.030952550, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1484, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.039115855, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000026358 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.869 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 580016 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2912, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2889, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 25321283, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1034388, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.240739315, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.073377306, 2024-05-17T04:55:27.870 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.095622052, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.064079548, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2888, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.080655078, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000027927 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 665376, 2024-05-17T04:55:27.871 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 241296 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8449, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.177902115, 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000021055 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 118, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 44, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 74, 2024-05-17T04:55:27.874 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 56, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 4, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-05-17T04:55:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:27.876 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-05-17T04:55:27.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-05-17T04:55:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.879 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 58310, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 67, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 6, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-05-17T04:55:27.880 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 70050511, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 70024025, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 58310 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 95, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 522, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 427, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 13, 2024-05-17T04:55:27.881 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 26, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 13, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 118, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 528, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 410, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 128632, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215440 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-05-17T04:55:27.882 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.883 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.885 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.886 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-05-17T04:55:27.887 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.888 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.890 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:27.891 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.892 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.893 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 1, 2024-05-17T04:55:27.894 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 2429, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:27.895 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:27.896 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:27.897 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 235518, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 83, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 170, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:27.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 300301, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 755, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 4910, 2024-05-17T04:55:27.899 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 4910, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4910, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.858945947, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000582270 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4910, 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 4911 2024-05-17T04:55:27.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 4910, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 4910, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 2615, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 2293, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-05-17T04:55:27.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-05-17T04:55:27.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-05-17T04:55:27.903 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 89, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 10, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 11, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 1, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-05-17T04:55:27.904 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:27.905 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 5329, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 67374373, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:27.906 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 5329, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5329, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 67374373, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:27.907 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4910, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 65856212, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4910, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 65856212, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4910, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 4910, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4910, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4910, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.909 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x557380d84c30": { 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:27.911 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:27.912 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-05-17T04:55:28.314 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:28 smithi105 ceph-mon[34795]: pgmap v1013: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-05-17T04:55:28.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:28 smithi110 ceph-mon[35497]: pgmap v1013: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-05-17T04:55:28.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:28 smithi050 ceph-mon[31739]: pgmap v1013: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-05-17T04:55:28.688 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-17T04:55:28.688 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-17T04:55:28.688 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-17T04:55:28.688 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-17T04:55:28.688 DEBUG:tasks.check_counter:Getting stats from h 2024-05-17T04:55:28.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-17T04:55:28.938 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:55:30.463 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:30 smithi105 ceph-mon[34795]: pgmap v1014: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-17T04:55:30.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:30 smithi110 ceph-mon[35497]: pgmap v1014: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-17T04:55:30.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:30 smithi050 ceph-mon[31739]: pgmap v1014: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1018, 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 995, 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 5009917, 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 431295, 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-05-17T04:55:30.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.093150660, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.033123762, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.047460429, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.016534273, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 994, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.029512456, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000029690 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 664992, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 240176 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2955, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 3248, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 10301766, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1518649, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.241312215, 2024-05-17T04:55:30.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.085772344, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.061090058, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.074818371, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3246, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.080290490, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000024735 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.783 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 579216 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1008, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1008, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 4452011, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 319735, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.068596846, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.018039984, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.021233348, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.023557998, 2024-05-17T04:55:30.784 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1007, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.023484057, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000023320 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-05-17T04:55:30.785 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7678, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.198099099, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000025800 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:30.786 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 1005, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 668, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 337, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 53, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-05-17T04:55:30.788 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 9, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-05-17T04:55:30.789 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 999, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1182610702, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:30.790 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.791 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:30.792 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-05-17T04:55:30.793 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 6929, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 88, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 21828374, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 22020978, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 22018990, 2024-05-17T04:55:30.794 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 6929 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 823, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1828, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1005, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 66, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 77, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 11, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 1005, 2024-05-17T04:55:30.795 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1881, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 876, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 122836, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215440 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.796 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-05-17T04:55:30.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.798 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.799 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.801 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.804 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.805 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-05-17T04:55:30.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.807 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 3, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-05-17T04:55:30.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 2432, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:30.809 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 207106, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 63, 2024-05-17T04:55:30.811 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 170, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 2531157, 2024-05-17T04:55:30.812 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 7778, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 4559, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 26, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 4, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 4559, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4559, 2024-05-17T04:55:30.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.246957297, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000492861 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4559, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 4560 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 4559, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 4559, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 2716, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-05-17T04:55:30.814 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1840, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 2, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:30.815 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-05-17T04:55:30.816 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 89, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2024-05-17T04:55:30.817 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 1, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-05-17T04:55:30.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 4977, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 19291445, 2024-05-17T04:55:30.819 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 4977, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4977, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 19291445, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.820 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4559, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 17828174, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4559, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 17828174, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.821 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4559, 2024-05-17T04:55:30.822 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 4559, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4559, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4559, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:30.823 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x562d9840cc30": { 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-05-17T04:55:30.824 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:30.825 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-05-17T04:55:31.614 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-17T04:55:31.614 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-17T04:55:31.614 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-17T04:55:31.614 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-17T04:55:31.614 DEBUG:tasks.check_counter:Getting stats from k 2024-05-17T04:55:31.615 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-17T04:55:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.b/config 2024-05-17T04:55:32.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:32 smithi105 ceph-mon[34795]: pgmap v1015: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-05-17T04:55:32.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:32 smithi110 ceph-mon[35497]: pgmap v1015: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-05-17T04:55:32.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:32 smithi050 ceph-mon[31739]: pgmap v1015: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-05-17T04:55:33.675 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-05-17T04:55:33.675 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:33.675 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 0, 2024-05-17T04:55:33.675 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2, 2024-05-17T04:55:33.675 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 0, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 0, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001019213, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.000211526, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:33.676 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 395, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 392, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 564256, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 242672, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.043824209, 2024-05-17T04:55:33.677 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.019828523, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.031629833, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 391, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.014477720, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000037027 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.678 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 564256, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 242672 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 6, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 302, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3264, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 210880, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.014666226, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.010766769, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001561586, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:33.679 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 301, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.012698869, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000042188 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 210880 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-05-17T04:55:33.680 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:33.681 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:33.682 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:33.683 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 148682, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 29, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:33.684 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 397, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 508397, 2024-05-17T04:55:33.685 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 397, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 397, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 508397, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-05-17T04:55:33.686 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-05-17T04:55:34.448 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:34 smithi105 ceph-mon[34795]: pgmap v1016: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-17T04:55:34.483 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-17T04:55:34.483 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-17T04:55:34.483 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-17T04:55:34.483 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-17T04:55:34.483 DEBUG:tasks.check_counter:Getting stats from c 2024-05-17T04:55:34.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-17T04:55:34.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:34 smithi110 ceph-mon[35497]: pgmap v1016: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-17T04:55:34.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:34 smithi050 ceph-mon[31739]: pgmap v1016: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-17T04:55:34.813 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:55:36.501 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:36 smithi110 ceph-mon[35497]: pgmap v1017: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:36.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:36 smithi105 ceph-mon[34795]: pgmap v1017: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:36.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:36 smithi050 ceph-mon[31739]: pgmap v1017: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-17T04:55:36.843 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-17T04:55:36.843 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:36.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 976669, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 931332, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 243395364, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 248698035, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 9, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 44.526002623, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 19.180810663, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 30.466311470, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.543724794, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 931330, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 13.942255259, 2024-05-17T04:55:36.844 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000014970 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1860837, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.262958149, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000141 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 33311, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 33287, 2024-05-17T04:55:36.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 8768658, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 48483479, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 9, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 2.651071410, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.903492444, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.644521169, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.936461195, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 33287, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.746464167, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000022425 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 681216, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 247872 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 575798, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 942395, 2024-05-17T04:55:36.847 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 185268271, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 383522712, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 13, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 9, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 33.447707007, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 14.736004391, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 20.562300999, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.224304368, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 942393, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 14.310813437, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000015185 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.848 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1467681, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.236129741, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000160 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 583536 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-17T04:55:36.849 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 115282, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 40.269859910, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000349316 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 98, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.009774850, 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000099743 2024-05-17T04:55:36.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 184518, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 184053, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 184053, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 363.088488490, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001972738 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 465, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:36.851 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 115, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 6, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 94, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 15, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 107, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 434, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 88, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 88, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 7, 2024-05-17T04:55:36.852 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 367910, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 366988, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 465, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 36, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 191, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-05-17T04:55:36.853 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 12, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 414, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 14, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 480, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1000, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1182528558, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:36.854 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 29, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 382297, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 309600, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 1, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 383754, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 57239, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 7, 2024-05-17T04:55:36.855 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 127 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 12, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 294, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 161, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 261, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 16, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 23, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 23, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-05-17T04:55:36.856 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 15, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-17T04:55:36.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 58311, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 58311, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 67, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 67, 2024-05-17T04:55:36.858 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 6, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 70050572, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 57593, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 161.452603239, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002803337 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-17T04:55:36.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 118, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 550, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 432, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 51, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 156, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 105, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 115, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 572, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 457, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 88, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 510, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 422, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 133768, 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 223632 2024-05-17T04:55:36.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 331089, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 108, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 184518, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 30, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 1251, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 32, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.208545182, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.006517036 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-17T04:55:36.861 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 181255, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 358.340795235, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001976998 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.862 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1862, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 3.480929468, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001869457 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.865 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-17T04:55:36.865 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.865 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.865 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-17T04:55:36.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 872, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.862159179, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000988714 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.006440774, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000920110 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.867 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 24, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.186425284, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.007767720 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.868 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.869 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.003193368, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003193368 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.870 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:36.872 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 6025, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 6025, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1181, 2024-05-17T04:55:36.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:36.874 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:36.875 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 299948, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 128, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:36.876 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 438133, 2024-05-17T04:55:36.877 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 1919, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 57959, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 65903138, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 33, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 57959, 2024-05-17T04:55:36.878 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 57959, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 160.670422897, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002772139 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 57959, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 58077 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 57959, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 14, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 57945, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-17T04:55:36.879 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 14, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 57607, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 86, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 73, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:36.880 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 1, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-17T04:55:36.881 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-17T04:55:36.882 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 89, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 2, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 226, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 40 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-05-17T04:55:36.883 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 12, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 6, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 46, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 23, 2024-05-17T04:55:36.884 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1585774, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 294686512, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1585774, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1585774, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 294686512, 2024-05-17T04:55:36.885 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:36.886 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 57959, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 65905645, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 57959, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 65905645, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:36.887 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 57959, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 57959, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 57959, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 57959, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.888 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 23, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2507, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 23, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 14, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2507, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.889 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55bcbf82a820": { 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 58311, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 65856268, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 58311, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:36.890 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 57593, 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 65856268, 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:36.891 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-17T04:55:37.619 INFO:tasks.check_counter:Daemon mds.c mds.exported=12 2024-05-17T04:55:37.620 INFO:tasks.check_counter:Daemon mds.c mds.imported=14 2024-05-17T04:55:37.620 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=12 2024-05-17T04:55:37.620 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=294 2024-05-17T04:55:37.620 DEBUG:tasks.check_counter:Getting stats from f 2024-05-17T04:55:37.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-17T04:55:37.860 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:55:38.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:38 smithi105 ceph-mon[34795]: pgmap v1018: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-17T04:55:38.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:38 smithi110 ceph-mon[35497]: pgmap v1018: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-17T04:55:38.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:38 smithi050 ceph-mon[31739]: pgmap v1018: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-17T04:55:39.805 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1788, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1765, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1125568, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 677600, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.163776281, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.057499340, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.060239003, 2024-05-17T04:55:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.041405709, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1764, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.065361493, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000037053 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 665920, 2024-05-17T04:55:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 242576 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 230, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 526, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 56472, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 743990, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.031473917, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015052268, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.005825368, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.007023007, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 524, 2024-05-17T04:55:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.019585777, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000037377 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 671424 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2780, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2780, 2024-05-17T04:55:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 10930056, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 883223, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.176806219, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.052290175, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.049071371, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.065175986, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2779, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.066149236, 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000023803 2024-05-17T04:55:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7403, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.173813289, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000023478 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 444, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 69, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 375, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 345, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 7, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-05-17T04:55:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-05-17T04:55:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-05-17T04:55:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-17T04:55:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-17T04:55:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 7816, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 10, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 14414699, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 14407279, 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-17T04:55:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 7816 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 162, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 684, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 522, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 24, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 64, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 40, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 444, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 908, 2024-05-17T04:55:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 464, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 123332, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 215440 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-05-17T04:55:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-17T04:55:39.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.829 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.832 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:39.833 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-17T04:55:39.834 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2431, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:39.835 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:39.836 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:39.837 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 190951, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 54, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2024-05-17T04:55:39.838 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 692305, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 8777, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 4372, 2024-05-17T04:55:39.839 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 4372, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4372, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2.023966772, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000462938 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4372, 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 4373 2024-05-17T04:55:39.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 4372, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 4372, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 2647, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 1723, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-17T04:55:39.841 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:39.842 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-17T04:55:39.843 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 89, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 2, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 5, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 7, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 2, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-17T04:55:39.844 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-05-17T04:55:39.845 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 4794, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 11656112, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 4794, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:39.846 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4794, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 11656112, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:39.847 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 4372, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 10221288, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4372, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 10221288, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:39.848 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 4372, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 4372, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4372, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 4372, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-17T04:55:39.850 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x557919526c30": { 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:39.851 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:39.852 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-17T04:55:40.562 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:40 smithi110 ceph-mon[35497]: pgmap v1019: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-17T04:55:40.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:40 smithi105 ceph-mon[34795]: pgmap v1019: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-17T04:55:40.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:40 smithi050 ceph-mon[31739]: pgmap v1019: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-17T04:55:40.589 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-17T04:55:40.589 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-17T04:55:40.589 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-17T04:55:40.590 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-17T04:55:40.590 DEBUG:tasks.check_counter:Getting stats from i 2024-05-17T04:55:40.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-17T04:55:40.827 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:55:42.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:42 smithi105 ceph-mon[34795]: pgmap v1020: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T04:55:42.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:42 smithi110 ceph-mon[35497]: pgmap v1020: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T04:55:42.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:42 smithi050 ceph-mon[31739]: pgmap v1020: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 517, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 519, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 153991, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 247363, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 10, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-05-17T04:55:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.045303968, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.013627521, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.012084574, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.014908993, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 516, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.029569797, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000057305 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 354851, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 575009, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 94761389, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 210092607, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 13, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 10, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 24.600832404, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 11.086571370, 2024-05-17T04:55:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 15.877247325, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.059544386, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 575008, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 10.291952725, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000017898 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 921425, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.157651256, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000171 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 683040, 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 248416 2024-05-17T04:55:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 156061, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 299987, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 32310205, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 81821567, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 11.517714145, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 4.894548597, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 7.206959200, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.008509790, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 299987, 2024-05-17T04:55:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 5.057789407, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000016860 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 454617, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.091418897, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000201 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 584416 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-17T04:55:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3827, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.785599847, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000466579 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-17T04:55:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 367, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.016962965, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000046220 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 89125, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 88098, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 88098, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 91.053456660, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001033547 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-17T04:55:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 1027, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 361, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 4, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 303, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 54, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 354, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 538, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 153, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 153, 2024-05-17T04:55:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 10, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 177473, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 174335, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 1026, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 67, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 9, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 990, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 36, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 424, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 41, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 540, 2024-05-17T04:55:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1000, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1182528558, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-17T04:55:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 16, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 108480, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 74242, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 1, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 114648, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 4741, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 31, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 765 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 92, 2024-05-17T04:55:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 140, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 80, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 140, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 103, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 192, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 88, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 104, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-17T04:55:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 36, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 7816, 2024-05-17T04:55:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 7816, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 10, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 10, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 14414699, 2024-05-17T04:55:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1757, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 5.121508211, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002914916 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 179, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 746, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 567, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 59, 2024-05-17T04:55:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 107, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 48, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 361, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 1005, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 644, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 153, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 650, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 497, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 131696, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 223632 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 164300, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 890, 2024-05-17T04:55:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 89125, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 35, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 2283, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 128, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.320528143, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002504126 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 82359, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 85.420111330, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001037167 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-17T04:55:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 104, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.334211133, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003213568 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2924, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.981869807, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000677794 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-17T04:55:42.657 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2253, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.224104786, 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000543322 2024-05-17T04:55:42.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 29, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.022326076, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000769864 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 194, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.444641526, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.007446605 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.659 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-17T04:55:42.660 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002870619, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002870619 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.661 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.662 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 106, 2024-05-17T04:55:42.663 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.302793240, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002856540 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-05-17T04:55:42.664 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 4014, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 4014, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1187, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:42.665 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:42.666 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:42.667 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 306209, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 117, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2024-05-17T04:55:42.668 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 793553, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 11286, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-17T04:55:42.669 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 2457, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 10296373, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 28, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 2457, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2457, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7.268882096, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002958437 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2457, 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2796 2024-05-17T04:55:42.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 2457, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 48, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 2409, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 48, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 1805, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 111, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-17T04:55:42.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 190, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-17T04:55:42.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 1, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-17T04:55:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 89, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 2, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 467, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 144 2024-05-17T04:55:42.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 16, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-17T04:55:42.675 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 8, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 176, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 88, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 511425, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 81171265, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 511425, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:42.676 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 511425, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 81171265, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:42.677 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2457, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 10305965, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2457, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 10305965, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:42.678 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2457, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2457, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2457, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2457, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 88, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 9592, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 88, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 48, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 9592, 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:42.680 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x5611a0490820": { 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 7816, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 10220395, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 7816, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:42.681 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1757, 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 10220395, 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:42.682 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-17T04:55:43.413 INFO:tasks.check_counter:Daemon mds.i mds.exported=36 2024-05-17T04:55:43.413 INFO:tasks.check_counter:Daemon mds.i mds.imported=41 2024-05-17T04:55:43.413 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=92 2024-05-17T04:55:43.413 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=140 2024-05-17T04:55:43.413 DEBUG:tasks.check_counter:Getting stats from l 2024-05-17T04:55:43.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-17T04:55:43.661 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.c/config 2024-05-17T04:55:44.548 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:44 smithi110 ceph-mon[35497]: pgmap v1021: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:55:44.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:44 smithi105 ceph-mon[34795]: pgmap v1021: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:55:44.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:44 smithi050 ceph-mon[31739]: pgmap v1021: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 404, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 399, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 568656, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 245984, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-05-17T04:55:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.046388257, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.020857593, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.031755051, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 399, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.016099460, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000040349 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 568656, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 245984 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 303, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 210960, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.014835192, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.010945292, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001432720, 2024-05-17T04:55:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 301, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.011991043, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000039837 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 210960 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.000237927, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000162610, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-17T04:55:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-17T04:55:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 152712, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 30, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-17T04:55:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-17T04:55:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 402, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 509150, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 402, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 402, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 509150, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-17T04:55:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-17T04:55:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-17T04:55:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-17T04:55:45.464 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-17T04:55:46.131 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-17T04:55:46.131 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-17T04:55:46.131 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-17T04:55:46.131 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-17T04:55:46.131 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-17T04:55:46.141 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-17T04:55:46.141 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:55:46.141 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T04:55:46.170 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:55:46.170 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T04:55:46.229 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T04:55:46.538 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:46.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:46 smithi105 ceph-mon[34795]: pgmap v1022: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T04:55:46.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:46 smithi110 ceph-mon[35497]: pgmap v1022: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T04:55:46.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:46 smithi050 ceph-mon[31739]: pgmap v1022: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-17T04:55:48.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 -- 172.21.15.50:0/3534284206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0102e00 msgr2=0x7fb0a0103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:48.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/3534284206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0102e00 0x7fb0a0103210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb09000b0a0 tx=0x7fb09000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 -- 172.21.15.50:0/3534284206 shutdown_connections 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/3534284206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 0x7fb0a010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/3534284206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0102e00 0x7fb0a0103210 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/3534284206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0107730 0x7fb0a0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 -- 172.21.15.50:0/3534284206 >> 172.21.15.50:0/3534284206 conn(0x7fb0a0079bd0 msgr2=0x7fb0a0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 -- 172.21.15.50:0/3534284206 shutdown_connections 2024-05-17T04:55:48.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.298+0000 7fb0a48e2700 1 -- 172.21.15.50:0/3534284206 wait complete. 2024-05-17T04:55:48.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 Processor -- start 2024-05-17T04:55:48.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 -- start start 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0102e00 0x7fb0a019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 0x7fb0a019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0112b60 con 0x7fb0a0107730 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01129e0 con 0x7fb0a0102e00 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb0a48e2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0112ce0 con 0x7fb0a0108080 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.299+0000 7fb09dd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:59672/0 (socket says 172.21.15.50:59672) 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 -- 172.21.15.50:0/4158883890 learned_addr learned my addr 172.21.15.50:0/4158883890 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09e59c700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0102e00 0x7fb0a019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09ed9d700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 0x7fb0a019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 -- 172.21.15.50:0/4158883890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 msgr2=0x7fb0a019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:48.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 0x7fb0a019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 -- 172.21.15.50:0/4158883890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0102e00 msgr2=0x7fb0a019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0102e00 0x7fb0a019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 -- 172.21.15.50:0/4158883890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a019f680 con 0x7fb0a0107730 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb09dd9b700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb08800d980 tx=0x7fb08800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088017070 con 0x7fb0a0107730 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb08800fd20 con 0x7fb0a0107730 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a019f910 con 0x7fb0a0107730 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08800e3b0 con 0x7fb0a0107730 2024-05-17T04:55:48.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.300+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a01a7290 con 0x7fb0a0107730 2024-05-17T04:55:48.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.301+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a0103210 con 0x7fb0a0107730 2024-05-17T04:55:48.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.303+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb08800f890 con 0x7fb0a0107730 2024-05-17T04:55:48.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.304+0000 7fb0977fe700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 0x7fb08c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:48.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.304+0000 7fb09e59c700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 0x7fb08c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:48.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.304+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb0880e1fc0 con 0x7fb0a0107730 2024-05-17T04:55:48.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.304+0000 7fb09e59c700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 0x7fb08c0b4e10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb090009900 tx=0x7fb09000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:48.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.307+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0880a99d0 con 0x7fb0a0107730 2024-05-17T04:55:48.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.537+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb0a002ea80 con 0x7fb0a0107730 2024-05-17T04:55:48.538 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:48 smithi050 ceph-mon[31739]: pgmap v1023: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.538+0000 7fb0977fe700 1 -- 172.21.15.50:0/4158883890 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7fb0880a92d0 con 0x7fb0a0107730 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/175302762 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3670657062 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3676423988 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:48.539 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:48.540 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2016476449 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:48.540 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3137083223 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:48.540 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3234552794 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 msgr2=0x7fb08c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 0x7fb08c0b4e10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb090009900 tx=0x7fb09000b700 comp rx=0 tx=0).stop 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 msgr2=0x7fb0a019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb08800d980 tx=0x7fb08800dc90 comp rx=0 tx=0).stop 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 shutdown_connections 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb0a0108080 0x7fb0a019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a0102e00 0x7fb0a019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fb08c0b2900 0x7fb08c0b4e10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 --2- 172.21.15.50:0/4158883890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0a0107730 0x7fb0a019ad50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:48.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 >> 172.21.15.50:0/4158883890 conn(0x7fb0a0079bd0 msgr2=0x7fb0a006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:48.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 shutdown_connections 2024-05-17T04:55:48.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:48.540+0000 7fb0a48e2700 1 -- 172.21.15.50:0/4158883890 wait complete. 2024-05-17T04:55:48.542 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-05-17T04:55:48.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:48 smithi105 ceph-mon[34795]: pgmap v1023: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:48.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:48 smithi110 ceph-mon[35497]: pgmap v1023: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:49.298 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-17T04:55:49.299 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T04:55:49.318 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-17T04:55:49.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:49 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/4158883890' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:49.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:49 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/4158883890' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:49.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:49 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/4158883890' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:49.613 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:55:50.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:50 smithi105 ceph-mon[34795]: pgmap v1024: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:55:50.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:50 smithi110 ceph-mon[35497]: pgmap v1024: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:55:50.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:50 smithi050 ceph-mon[31739]: pgmap v1024: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-17T04:55:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3095505146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c58107750 msgr2=0x7f6c58107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3095505146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c58107750 0x7f6c58107b60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009990 tx=0x7f6c48009ca0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3095505146 shutdown_connections 2024-05-17T04:55:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3095505146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c581080a0 0x7f6c5810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3095505146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58102e70 0x7f6c58103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3095505146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c58107750 0x7f6c58107b60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3095505146 >> 172.21.15.50:0/3095505146 conn(0x7f6c58079ae0 msgr2=0x7f6c58079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3095505146 shutdown_connections 2024-05-17T04:55:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.423+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3095505146 wait complete. 2024-05-17T04:55:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 Processor -- start 2024-05-17T04:55:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 -- start start 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c58102e70 0x7f6c581a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58107750 0x7f6c581a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58112bb0 con 0x7f6c581080a0 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58112a30 con 0x7f6c58102e70 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5fe71700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58112d30 con 0x7f6c58107750 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5e40e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5e40e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:51578/0 (socket says 172.21.15.50:51578) 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5e40e700 1 -- 172.21.15.50:0/3073632871 learned_addr learned my addr 172.21.15.50:0/3073632871 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.424+0000 7f6c5d40c700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58107750 0x7f6c581a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5dc0d700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c58102e70 0x7f6c581a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 -- 172.21.15.50:0/3073632871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58107750 msgr2=0x7f6c581a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58107750 0x7f6c581a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 -- 172.21.15.50:0/3073632871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c58102e70 msgr2=0x7f6c581a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c58102e70 0x7f6c581a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 -- 172.21.15.50:0/3073632871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5819cf00 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5e40e700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c50007ee0 tx=0x7f6c5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000dd20 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c50004020 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5819d1f0 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50015660 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.425+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c581a93f0 con 0x7f6c581080a0 2024-05-17T04:55:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.426+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c581042d0 con 0x7f6c581080a0 2024-05-17T04:55:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.429+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6c500041d0 con 0x7f6c581080a0 2024-05-17T04:55:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.430+0000 7f6c4effd700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 0x7f6c440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:55:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.430+0000 7f6c5dc0d700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 0x7f6c440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:55:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.430+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6c500e1f30 con 0x7f6c581080a0 2024-05-17T04:55:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.430+0000 7f6c5dc0d700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 0x7f6c440b4d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c5819db00 tx=0x7f6c5400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:55:51.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.433+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c50010070 con 0x7f6c581080a0 2024-05-17T04:55:51.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.667+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c58103280 con 0x7f6c581080a0 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.668+0000 7f6c4effd700 1 -- 172.21.15.50:0/3073632871 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7f6c500a9860 con 0x7f6c581080a0 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/175302762 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/2489127888 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3670657062 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3676423988 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6801/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:6800/193225580 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/2016476449 2024-05-18T04:24:18.593378+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3137083223 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:51.669 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50:0/3234552794 2024-05-18T04:23:51.164359+0000 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 msgr2=0x7f6c440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 0x7f6c440b4d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c5819db00 tx=0x7f6c5400b410 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 msgr2=0x7f6c5819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c50007ee0 tx=0x7f6c5000d3b0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 shutdown_connections 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c440b2880 0x7f6c440b4d90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c581080a0 0x7f6c5819c7d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c58102e70 0x7f6c581a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 --2- 172.21.15.50:0/3073632871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c58107750 0x7f6c581a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:55:51.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 >> 172.21.15.50:0/3073632871 conn(0x7f6c58079ae0 msgr2=0x7f6c5810e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:55:51.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 shutdown_connections 2024-05-17T04:55:51.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:55:51.671+0000 7f6c5fe71700 1 -- 172.21.15.50:0/3073632871 wait complete. 2024-05-17T04:55:51.673 INFO:teuthology.orchestra.run.smithi050.stderr:listed 10 entries 2024-05-17T04:55:52.349 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi050.front.sepia.ceph.com... 2024-05-17T04:55:52.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:55:52.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo fusermount -u /tmp/cephfs 2024-05-17T04:55:52.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi050.stderr:ceph-fuse[87745]: fuse finished with error 0 and tester_r 0 2024-05-17T04:55:52.434 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T04:55:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:52 smithi105 ceph-mon[34795]: pgmap v1025: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:55:52.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:52 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3073632871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:52 smithi110 ceph-mon[35497]: pgmap v1025: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:55:52.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:52 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3073632871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:52 smithi050 ceph-mon[31739]: pgmap v1025: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-17T04:55:52.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:52 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3073632871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-17T04:55:54.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:54 smithi105 ceph-mon[34795]: pgmap v1026: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:54.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:54 smithi110 ceph-mon[35497]: pgmap v1026: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:54.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:54 smithi050 ceph-mon[31739]: pgmap v1026: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-17T04:55:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:56 smithi105 ceph-mon[34795]: pgmap v1027: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-05-17T04:55:56.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:56 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:55:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:56 smithi110 ceph-mon[35497]: pgmap v1027: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-05-17T04:55:56.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:56 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:55:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:56 smithi050 ceph-mon[31739]: pgmap v1027: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-05-17T04:55:56.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:56 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-17T04:55:58.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: pgmap v1028: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:55:58.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:55:57 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: pgmap v1028: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:55:58.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:55:57 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: pgmap v1028: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-17T04:55:58.076 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:55:57 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' 2024-05-17T04:55:58.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T04:55:58.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:55:58.536 DEBUG:teuthology.orchestra.run.smithi050:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-17T04:55:58.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:55:58.566 DEBUG:teuthology.orchestra.run.smithi050:> ip netns list 2024-05-17T04:55:58.624 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-17T04:55:58.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:55:58.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-17T04:55:58.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T04:55:58.696 DEBUG:teuthology.orchestra.run.smithi050:> sudo ip link delete ceph-brx 2024-05-17T04:55:58.787 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-17T04:55:58.797 INFO:tasks.cephadm:Teardown begin 2024-05-17T04:55:58.798 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T04:55:58.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:55:59.054 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:00.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:00 smithi050 ceph-mon[31739]: pgmap v1029: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:56:00.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:00 smithi105 ceph-mon[34795]: pgmap v1029: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:56:00.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:00 smithi110 ceph-mon[35497]: pgmap v1029: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-05-17T04:56:01.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 -- 172.21.15.50:0/511012330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701091f0 msgr2=0x7f5e70110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 --2- 172.21.15.50:0/511012330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701091f0 0x7f5e70110b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800b0a0 tx=0x7f5e6800b3b0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 -- 172.21.15.50:0/511012330 shutdown_connections 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 --2- 172.21.15.50:0/511012330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701091f0 0x7f5e70110b50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 --2- 172.21.15.50:0/511012330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 0x7f5e70102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 --2- 172.21.15.50:0/511012330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701088a0 0x7f5e70108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 -- 172.21.15.50:0/511012330 >> 172.21.15.50:0/511012330 conn(0x7f5e700fe1a0 msgr2=0x7f5e701005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 -- 172.21.15.50:0/511012330 shutdown_connections 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.038+0000 7f5e77f13700 1 -- 172.21.15.50:0/511012330 wait complete. 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 Processor -- start 2024-05-17T04:56:01.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 -- start start 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 0x7f5e7019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 0x7f5e7019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70113d00 con 0x7f5e701088a0 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70113b80 con 0x7f5e70102480 2024-05-17T04:56:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e77f13700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70113e80 con 0x7f5e701091f0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e754ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.039+0000 7f5e754ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55902/0 (socket says 172.21.15.50:55902) 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 -- 172.21.15.50:0/2026421628 learned_addr learned my addr 172.21.15.50:0/2026421628 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e75caf700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 0x7f5e7019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e764b0700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 0x7f5e7019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 -- 172.21.15.50:0/2026421628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 msgr2=0x7f5e7019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 0x7f5e7019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 -- 172.21.15.50:0/2026421628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 msgr2=0x7f5e7019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 0x7f5e7019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 -- 172.21.15.50:0/2026421628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7019f5b0 con 0x7f5e701088a0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e764b0700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 0x7f5e7019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e754ae700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000d950 tx=0x7f5e6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:01.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60017070 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e60009dc0 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7019f8a0 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000f3c0 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.040+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e701a71c0 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.041+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e70102890 con 0x7f5e701088a0 2024-05-17T04:56:01.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.044+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5e6001a070 con 0x7f5e701088a0 2024-05-17T04:56:01.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.045+0000 7f5e66ffd700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 0x7f5e5c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:01.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.045+0000 7f5e75caf700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 0x7f5e5c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:01.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.045+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5e600e1e30 con 0x7f5e701088a0 2024-05-17T04:56:01.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.045+0000 7f5e75caf700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 0x7f5e5c0b4d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005950 tx=0x7f5e6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:01.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.048+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e600a9760 con 0x7f5e701088a0 2024-05-17T04:56:01.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.310+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e7019ba40 con 0x7f5e701088a0 2024-05-17T04:56:01.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.310+0000 7f5e66ffd700 1 -- 172.21.15.50:0/2026421628 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11885 (secure 0 0 0) 0x7f5e600a9060 con 0x7f5e701088a0 2024-05-17T04:56:01.311 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:56:01.311 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:54:28.535960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24499,24506]},"id":1}]} 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 msgr2=0x7f5e5c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 0x7f5e5c0b4d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005950 tx=0x7f5e6c00b410 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 msgr2=0x7f5e7019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000d950 tx=0x7f5e6000dc60 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 shutdown_connections 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e701091f0 0x7f5e7019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e70102480 0x7f5e7019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f5e5c0b2880 0x7f5e5c0b4d90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 --2- 172.21.15.50:0/2026421628 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5e701088a0 0x7f5e7019ac80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 >> 172.21.15.50:0/2026421628 conn(0x7f5e700fe1a0 msgr2=0x7f5e701005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 shutdown_connections 2024-05-17T04:56:01.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:01.314+0000 7f5e77f13700 1 -- 172.21.15.50:0/2026421628 wait complete. 2024-05-17T04:56:01.315 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 45 2024-05-17T04:56:02.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-17T04:56:02.287 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:02.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:02 smithi050 ceph-mon[31739]: pgmap v1030: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T04:56:02.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:02 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/2026421628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:02 smithi105 ceph-mon[34795]: pgmap v1030: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T04:56:02.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:02 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/2026421628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:02 smithi110 ceph-mon[35497]: pgmap v1030: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-05-17T04:56:02.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:02 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/2026421628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 -- 172.21.15.50:0/2421604071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78681091a0 msgr2=0x7f7868110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 --2- 172.21.15.50:0/2421604071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78681091a0 0x7f7868110b00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f786000b0a0 tx=0x7f786000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 -- 172.21.15.50:0/2421604071 shutdown_connections 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 --2- 172.21.15.50:0/2421604071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78681091a0 0x7f7868110b00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 --2- 172.21.15.50:0/2421604071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7868103f20 0x7f7868104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 --2- 172.21.15.50:0/2421604071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 0x7f7868108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 -- 172.21.15.50:0/2421604071 >> 172.21.15.50:0/2421604071 conn(0x7f78680ff9c0 msgr2=0x7f7868101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 -- 172.21.15.50:0/2421604071 shutdown_connections 2024-05-17T04:56:04.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.092+0000 7f786d85f700 1 -- 172.21.15.50:0/2421604071 wait complete. 2024-05-17T04:56:04.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 Processor -- start 2024-05-17T04:56:04.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 -- start start 2024-05-17T04:56:04.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:04.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 0x7f786819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:04.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78681091a0 0x7f786819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7868113c90 con 0x7f7868103f20 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7868113b10 con 0x7f7868108850 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f786d85f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7868113e10 con 0x7f78681091a0 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55920/0 (socket says 172.21.15.50:55920) 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 -- 172.21.15.50:0/589136014 learned_addr learned my addr 172.21.15.50:0/589136014 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f78667fc700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 0x7f786819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f78677fe700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78681091a0 0x7f786819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 -- 172.21.15.50:0/589136014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78681091a0 msgr2=0x7f786819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78681091a0 0x7f786819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 -- 172.21.15.50:0/589136014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 msgr2=0x7f786819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 0x7f786819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.093+0000 7f7866ffd700 1 -- 172.21.15.50:0/589136014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786819f520 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f7866ffd700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9a0 tx=0x7f785800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800ca10 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f785800cbb0 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786819f7b0 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858018680 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.094+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681a7130 con 0x7f7868103f20 2024-05-17T04:56:04.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.095+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7868104330 con 0x7f7868103f20 2024-05-17T04:56:04.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.098+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7858010430 con 0x7f7868103f20 2024-05-17T04:56:04.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.099+0000 7f786c85d700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 0x7f78500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:04.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.099+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f78580e2ac0 con 0x7f7868103f20 2024-05-17T04:56:04.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.099+0000 7f78667fc700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 0x7f78500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:04.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.099+0000 7f78667fc700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 0x7f78500b4e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7868107300 tx=0x7f785c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:04.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.102+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78580aa450 con 0x7f7868103f20 2024-05-17T04:56:04.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:04 smithi050 ceph-mon[31739]: pgmap v1031: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-05-17T04:56:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.370+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786802eab0 con 0x7f7868103f20 2024-05-17T04:56:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.371+0000 7f786c85d700 1 -- 172.21.15.50:0/589136014 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11885 (secure 0 0 0) 0x7f78580a9d50 con 0x7f7868103f20 2024-05-17T04:56:04.371 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:56:04.371 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1993874157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1993874157},{"type":"v1","addr":"172.21.15.50:6835","nonce":1993874157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/2083526209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":2083526209},{"type":"v1","addr":"172.21.15.50:6837","nonce":2083526209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/3126760693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":3126760693},{"type":"v1","addr":"172.21.15.50:6839","nonce":3126760693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6841/3896696320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6840","nonce":3896696320},{"type":"v1","addr":"172.21.15.50:6841","nonce":3896696320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/32414548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":32414548},{"type":"v1","addr":"172.21.15.105:6833","nonce":32414548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3447375489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3447375489},{"type":"v1","addr":"172.21.15.110:6833","nonce":3447375489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-17T04:31:45.150094+0000","modified":"2024-05-17T04:54:28.535960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24499,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/1830970686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1830970686},{"type":"v1","addr":"172.21.15.105:6835","nonce":1830970686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/1276100312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1276100312},{"type":"v1","addr":"172.21.15.110:6835","nonce":1276100312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/1162023031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1162023031},{"type":"v1","addr":"172.21.15.105:6837","nonce":1162023031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.110:6837/2935546223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2935546223},{"type":"v1","addr":"172.21.15.110:6837","nonce":2935546223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.110:6839/2082916819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":2082916819},{"type":"v1","addr":"172.21.15.110:6839","nonce":2082916819}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/3722840784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3722840784},{"type":"v1","addr":"172.21.15.105:6839","nonce":3722840784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24487,24499,24506]},"id":1}]} 2024-05-17T04:56:04.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 msgr2=0x7f78500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:04.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 0x7f78500b4e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7868107300 tx=0x7f785c009260 comp rx=0 tx=0).stop 2024-05-17T04:56:04.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 msgr2=0x7f786819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9a0 tx=0x7f785800ecb0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 shutdown_connections 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78681091a0 0x7f786819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f78500b2900 0x7f78500b4e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7868103f20 0x7f786819a650 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 --2- 172.21.15.50:0/589136014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7868108850 0x7f786819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 >> 172.21.15.50:0/589136014 conn(0x7f78680ff9c0 msgr2=0x7f7868101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 shutdown_connections 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:04.374+0000 7f786d85f700 1 -- 172.21.15.50:0/589136014 wait complete. 2024-05-17T04:56:04.374 INFO:teuthology.orchestra.run.smithi050.stderr:dumped fsmap epoch 45 2024-05-17T04:56:04.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:04 smithi105 ceph-mon[34795]: pgmap v1031: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-05-17T04:56:04.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:04 smithi110 ceph-mon[35497]: pgmap v1031: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-05-17T04:56:05.060 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-17T04:56:05.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:05 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/589136014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:05.318 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:05.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:05 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/589136014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:05.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:05 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/589136014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-17T04:56:06.494 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:06 smithi050 ceph-mon[31739]: pgmap v1032: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T04:56:06.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:06 smithi105 ceph-mon[34795]: pgmap v1032: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T04:56:06.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:06 smithi110 ceph-mon[35497]: pgmap v1032: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/655645832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0102e00 msgr2=0x7f6ca0103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/655645832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0102e00 0x7f6ca0103210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009990 tx=0x7f6c90009ca0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/655645832 shutdown_connections 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/655645832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0108080 0x7f6ca010f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/655645832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0102e00 0x7f6ca0103210 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/655645832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0107730 0x7f6ca0107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/655645832 >> 172.21.15.50:0/655645832 conn(0x7f6ca0079bd0 msgr2=0x7f6ca0079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/655645832 shutdown_connections 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.117+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/655645832 wait complete. 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.118+0000 7f6ca5d6a700 1 Processor -- start 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.118+0000 7f6ca5d6a700 1 -- start start 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0102e00 0x7f6ca019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0108080 0x7f6ca019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0112bb0 con 0x7f6ca0107730 2024-05-17T04:56:07.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0112a30 con 0x7f6ca0102e00 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6ca5d6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0112d30 con 0x7f6ca0108080 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55950/0 (socket says 172.21.15.50:55950) 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 -- 172.21.15.50:0/1289689979 learned_addr learned my addr 172.21.15.50:0/1289689979 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9f7fe700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0102e00 0x7f6ca019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9ffff700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0108080 0x7f6ca019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 -- 172.21.15.50:0/1289689979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0108080 msgr2=0x7f6ca019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0108080 0x7f6ca019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 -- 172.21.15.50:0/1289689979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0102e00 msgr2=0x7f6ca019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0102e00 0x7f6ca019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.119+0000 7f6c9effd700 1 -- 172.21.15.50:0/1289689979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca019f5f0 con 0x7f6ca0107730 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6c9effd700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c8f0 tx=0x7f6c9400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c940078d0 con 0x7f6ca0107730 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c9400f450 con 0x7f6ca0107730 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94018660 con 0x7f6ca0107730 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca019f8e0 con 0x7f6ca0107730 2024-05-17T04:56:07.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.120+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a7200 con 0x7f6ca0107730 2024-05-17T04:56:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.122+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca0103210 con 0x7f6ca0107730 2024-05-17T04:56:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.123+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6c9400f900 con 0x7f6ca0107730 2024-05-17T04:56:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.123+0000 7f6c9cff9700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 0x7f6c880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.123+0000 7f6c9f7fe700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 0x7f6c880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:07.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.124+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6c940e2380 con 0x7f6ca0107730 2024-05-17T04:56:07.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.124+0000 7f6c9f7fe700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 0x7f6c880b4d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c9000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.128+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c94014030 con 0x7f6ca0107730 2024-05-17T04:56:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.345+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ca002ea80 con 0x7f6ca0107730 2024-05-17T04:56:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.347+0000 7f6c9cff9700 1 -- 172.21.15.50:0/1289689979 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24722 (secure 0 0 0) 0x7f6c940a9d10 con 0x7f6ca0107730 2024-05-17T04:56:07.347 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-17T04:56:07.347 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":89,"fsid":"0b3ae782-1405-11ef-bc9a-c7b262605968","created":"2024-05-17T04:23:23.371105+0000","modified":"2024-05-17T04:33:20.924661+0000","last_up_change":"2024-05-17T04:30:36.259873+0000","last_in_change":"2024-05-17T04:30:15.278301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T04:26:42.688129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T04:31:37.906052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T04:31:41.858212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-17T04:31:53.969356+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"126dad7f-7abd-4e39-8390-c63a919ff76f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6803","nonce":2573267330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6805","nonce":2573267330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6809","nonce":2573267330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2573267330},{"type":"v1","addr":"172.21.15.50:6807","nonce":2573267330}]},"public_addr":"172.21.15.50:6803/2573267330","cluster_addr":"172.21.15.50:6805/2573267330","heartbeat_back_addr":"172.21.15.50:6809/2573267330","heartbeat_front_addr":"172.21.15.50:6807/2573267330","state":["exists","up"]},{"osd":1,"uuid":"b5a2001e-c5a7-46d8-b1b5-7ee0e73c2481","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6801","nonce":722687301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6803","nonce":722687301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6807","nonce":722687301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":722687301},{"type":"v1","addr":"172.21.15.105:6805","nonce":722687301}]},"public_addr":"172.21.15.105:6801/722687301","cluster_addr":"172.21.15.105:6803/722687301","heartbeat_back_addr":"172.21.15.105:6807/722687301","heartbeat_front_addr":"172.21.15.105:6805/722687301","state":["exists","up"]},{"osd":2,"uuid":"ac2135de-eec8-43a3-bbeb-49c09e2b60e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6801","nonce":3787373914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6803","nonce":3787373914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6807","nonce":3787373914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3787373914},{"type":"v1","addr":"172.21.15.110:6805","nonce":3787373914}]},"public_addr":"172.21.15.110:6801/3787373914","cluster_addr":"172.21.15.110:6803/3787373914","heartbeat_back_addr":"172.21.15.110:6807/3787373914","heartbeat_front_addr":"172.21.15.110:6805/3787373914","state":["exists","up"]},{"osd":3,"uuid":"5004e0dd-7a4c-4229-abfe-72186a1ebf29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6811","nonce":3258879203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6813","nonce":3258879203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6817","nonce":3258879203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3258879203},{"type":"v1","addr":"172.21.15.50:6815","nonce":3258879203}]},"public_addr":"172.21.15.50:6811/3258879203","cluster_addr":"172.21.15.50:6813/3258879203","heartbeat_back_addr":"172.21.15.50:6817/3258879203","heartbeat_front_addr":"172.21.15.50:6815/3258879203","state":["exists","up"]},{"osd":4,"uuid":"2af1d73f-56c3-481a-b9ca-26dd81248baf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6809","nonce":908958206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6811","nonce":908958206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6815","nonce":908958206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":908958206},{"type":"v1","addr":"172.21.15.105:6813","nonce":908958206}]},"public_addr":"172.21.15.105:6809/908958206","cluster_addr":"172.21.15.105:6811/908958206","heartbeat_back_addr":"172.21.15.105:6815/908958206","heartbeat_front_addr":"172.21.15.105:6813/908958206","state":["exists","up"]},{"osd":5,"uuid":"7fec3a3b-bf8f-4065-8427-66d05c1a81a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6809","nonce":1542934887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6811","nonce":1542934887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6815","nonce":1542934887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1542934887},{"type":"v1","addr":"172.21.15.110:6813","nonce":1542934887}]},"public_addr":"172.21.15.110:6809/1542934887","cluster_addr":"172.21.15.110:6811/1542934887","heartbeat_back_addr":"172.21.15.110:6815/1542934887","heartbeat_front_addr":"172.21.15.110:6813/1542934887","state":["exists","up"]},{"osd":6,"uuid":"78770dde-1e76-4d87-b515-1127983aa2c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6819","nonce":159814107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6821","nonce":159814107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6825","nonce":159814107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":159814107},{"type":"v1","addr":"172.21.15.50:6823","nonce":159814107}]},"public_addr":"172.21.15.50:6819/159814107","cluster_addr":"172.21.15.50:6821/159814107","heartbeat_back_addr":"172.21.15.50:6825/159814107","heartbeat_front_addr":"172.21.15.50:6823/159814107","state":["exists","up"]},{"osd":7,"uuid":"7d035f38-f555-490f-b5bf-018aef1dbb18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6817","nonce":2725719637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6819","nonce":2725719637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6823","nonce":2725719637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2725719637},{"type":"v1","addr":"172.21.15.105:6821","nonce":2725719637}]},"public_addr":"172.21.15.105:6817/2725719637","cluster_addr":"172.21.15.105:6819/2725719637","heartbeat_back_addr":"172.21.15.105:6823/2725719637","heartbeat_front_addr":"172.21.15.105:6821/2725719637","state":["exists","up"]},{"osd":8,"uuid":"0df6cef8-8d82-478a-98a1-980848605ac9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6817","nonce":2523141158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6819","nonce":2523141158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6823","nonce":2523141158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":2523141158},{"type":"v1","addr":"172.21.15.110:6821","nonce":2523141158}]},"public_addr":"172.21.15.110:6817/2523141158","cluster_addr":"172.21.15.110:6819/2523141158","heartbeat_back_addr":"172.21.15.110:6823/2523141158","heartbeat_front_addr":"172.21.15.110:6821/2523141158","state":["exists","up"]},{"osd":9,"uuid":"23c1e66f-5c60-40f9-a478-e9a4e43517ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6827","nonce":595367012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6829","nonce":595367012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6833","nonce":595367012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":595367012},{"type":"v1","addr":"172.21.15.50:6831","nonce":595367012}]},"public_addr":"172.21.15.50:6827/595367012","cluster_addr":"172.21.15.50:6829/595367012","heartbeat_back_addr":"172.21.15.50:6833/595367012","heartbeat_front_addr":"172.21.15.50:6831/595367012","state":["exists","up"]},{"osd":10,"uuid":"d226622f-07c2-4a51-a54a-74440d67a19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6825","nonce":3101480579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6827","nonce":3101480579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6831","nonce":3101480579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3101480579},{"type":"v1","addr":"172.21.15.105:6829","nonce":3101480579}]},"public_addr":"172.21.15.105:6825/3101480579","cluster_addr":"172.21.15.105:6827/3101480579","heartbeat_back_addr":"172.21.15.105:6831/3101480579","heartbeat_front_addr":"172.21.15.105:6829/3101480579","state":["exists","up"]},{"osd":11,"uuid":"4ae7edc4-3891-4484-b0ca-d836265d6592","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6825","nonce":148686572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6827","nonce":148686572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6831","nonce":148686572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":148686572},{"type":"v1","addr":"172.21.15.110:6829","nonce":148686572}]},"public_addr":"172.21.15.110:6825/148686572","cluster_addr":"172.21.15.110:6827/148686572","heartbeat_back_addr":"172.21.15.110:6831/148686572","heartbeat_front_addr":"172.21.15.110:6829/148686572","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:25:52.507485+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:16.654130+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:26:40.364592+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:05.646831+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:31.251300+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:27:56.242976+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:22.997843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:28:48.804626+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:14.431433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:29:41.340588+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:08.006823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T04:30:33.721344+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.50:6801/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/175302762":"2024-05-18T04:24:18.593378+0000","172.21.15.50:6800/2489127888":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3670657062":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3676423988":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6801/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:6800/193225580":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/2016476449":"2024-05-18T04:24:18.593378+0000","172.21.15.50:0/3137083223":"2024-05-18T04:23:51.164359+0000","172.21.15.50:0/3234552794":"2024-05-18T04:23:51.164359+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 msgr2=0x7f6c880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 0x7f6c880b4d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006010 tx=0x7f6c9000b540 comp rx=0 tx=0).stop 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 msgr2=0x7f6ca019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c8f0 tx=0x7f6c9400cc00 comp rx=0 tx=0).stop 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 shutdown_connections 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ca0108080 0x7f6ca019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ca0102e00 0x7f6ca019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f6c880b2880 0x7f6c880b4d90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 --2- 172.21.15.50:0/1289689979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca0107730 0x7f6ca019acc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.349+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 >> 172.21.15.50:0/1289689979 conn(0x7f6ca0079bd0 msgr2=0x7f6ca006ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.350+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 shutdown_connections 2024-05-17T04:56:07.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:07.350+0000 7f6ca5d6a700 1 -- 172.21.15.50:0/1289689979 wait complete. 2024-05-17T04:56:08.078 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs fail cephfs 2024-05-17T04:56:08.333 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:08.456 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:08 smithi050 ceph-mon[31739]: pgmap v1033: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:56:08.456 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:08 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/1289689979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:56:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:08 smithi105 ceph-mon[34795]: pgmap v1033: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:56:08.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:08 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/1289689979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:56:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:08 smithi110 ceph-mon[35497]: pgmap v1033: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-17T04:56:08.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:08 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/1289689979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-17T04:56:10.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.115+0000 7f99acf86700 1 -- 172.21.15.50:0/3770688157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 msgr2=0x7f99a810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:10.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.115+0000 7f99acf86700 1 --2- 172.21.15.50:0/3770688157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a810dd10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f999c009990 tx=0x7f999c009ca0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.115+0000 7f99acf86700 1 -- 172.21.15.50:0/3770688157 shutdown_connections 2024-05-17T04:56:10.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 --2- 172.21.15.50:0/3770688157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 0x7f99a8110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 --2- 172.21.15.50:0/3770688157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 0x7f99a8105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 --2- 172.21.15.50:0/3770688157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a810dd10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 -- 172.21.15.50:0/3770688157 >> 172.21.15.50:0/3770688157 conn(0x7f99a8079ae0 msgr2=0x7f99a8079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 -- 172.21.15.50:0/3770688157 shutdown_connections 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 -- 172.21.15.50:0/3770688157 wait complete. 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 Processor -- start 2024-05-17T04:56:10.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.116+0000 7f99acf86700 1 -- start start 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 0x7f99a8102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 0x7f99a81035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8113c60 con 0x7f99a81057c0 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8113ae0 con 0x7f99a810e2a0 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99acf86700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8113de0 con 0x7f99a8104e70 2024-05-17T04:56:10.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42048/0 (socket says 172.21.15.50:42048) 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 -- 172.21.15.50:0/149119467 learned_addr learned my addr 172.21.15.50:0/149119467 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a659c700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 0x7f99a8102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a6d9d700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 0x7f99a81035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 -- 172.21.15.50:0/149119467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 msgr2=0x7f99a8102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 0x7f99a8102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 -- 172.21.15.50:0/149119467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 msgr2=0x7f99a81035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 0x7f99a81035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.117+0000 7f99a5d9b700 1 -- 172.21.15.50:0/149119467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a89f0 con 0x7f99a81057c0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99a5d9b700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c027070 con 0x7f99a81057c0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f999c00fcc0 con 0x7f99a81057c0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a8c80 con 0x7f99a81057c0 2024-05-17T04:56:10.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c022d70 con 0x7f99a81057c0 2024-05-17T04:56:10.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.118+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a9160 con 0x7f99a81057c0 2024-05-17T04:56:10.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.119+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8105280 con 0x7f99a81057c0 2024-05-17T04:56:10.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.120+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f999c02a070 con 0x7f99a81057c0 2024-05-17T04:56:10.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.121+0000 7f99977fe700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 0x7f99900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:10.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.121+0000 7f99a659c700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 0x7f99900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:10.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.121+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f999c0e35c0 con 0x7f99a81057c0 2024-05-17T04:56:10.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.122+0000 7f99a659c700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 0x7f99900b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f99980060b0 tx=0x7f999800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:10.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.124+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f999c013070 con 0x7f99a81057c0 2024-05-17T04:56:10.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.388+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f99a8104090 con 0x7f99a81057c0 2024-05-17T04:56:10.388 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:10 smithi050 ceph-mon[31739]: pgmap v1034: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-17T04:56:10.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.390+0000 7f99977fe700 1 -- 172.21.15.50:0/149119467 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f999c0aafe0 con 0x7f99a81057c0 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 msgr2=0x7f99900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 0x7f99900b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f99980060b0 tx=0x7f999800c040 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 msgr2=0x7f99a81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c0047b0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 shutdown_connections 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a810e2a0 0x7f99a81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99a8104e70 0x7f99a8102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f99900b2880 0x7f99900b4d90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 --2- 172.21.15.50:0/149119467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f99a81057c0 0x7f99a81030a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:10.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 >> 172.21.15.50:0/149119467 conn(0x7f99a8079ae0 msgr2=0x7f99a80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:10.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 shutdown_connections 2024-05-17T04:56:10.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:10.394+0000 7f99acf86700 1 -- 172.21.15.50:0/149119467 wait complete. 2024-05-17T04:56:10.395 INFO:teuthology.orchestra.run.smithi050.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T04:56:10.500 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:10 smithi110 ceph-mon[35497]: pgmap v1034: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-17T04:56:10.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:10 smithi105 ceph-mon[34795]: pgmap v1034: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:56:10 smithi105 ceph-mds[64946]: -- 172.21.15.105:0/2951040739 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557788f65a40 con 0x557788fab600 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: starting mds.b at 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.565 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:56:10 smithi105 ceph-mds[64334]: -- 172.21.15.105:0/2417682267 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af93925a40 con 0x55af9396ba80 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: starting mds.e at 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:56:10 smithi105 ceph-mds[64639]: -- 172.21.15.105:0/2232883130 <== mon.0 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564351613a40 con 0x564351659180 2024-05-17T04:56:10.566 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:56:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: starting mds.h at 2024-05-17T04:56:10.722 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:10 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a[31735]: 2024-05-17T04:56:10.389+0000 7fc48d55c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T04:56:10.817 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.817 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.817 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:56:10 smithi110 ceph-mds[66654]: -- 172.21.15.110:0/2830938918 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1d361da40 con 0x55c1d3663a80 2024-05-17T04:56:10.817 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: starting mds.i at 2024-05-17T04:56:10.817 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:56:10 smithi110 ceph-mds[66352]: -- 172.21.15.110:0/3376424194 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b71c47a40 con 0x561b71c8d600 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: starting mds.c at 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: ignoring --setuser ceph since I am not root 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: ignoring --setgroup ceph since I am not root 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:56:10 smithi110 ceph-mds[66052]: -- 172.21.15.110:0/2632730423 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c50a4b5a40 con 0x55c50a4fb180 2024-05-17T04:56:10.818 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:56:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: starting mds.f at 2024-05-17T04:56:11.099 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T04:56:11.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:11 smithi050 ceph-mon[31739]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T04:56:11.360 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:11.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T04:56:11.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T04:56:11.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T04:56:11.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T04:56:11.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T04:56:11.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:11 smithi105 ceph-mon[34795]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T04:56:11.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-17T04:56:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-17T04:56:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T04:56:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/149119467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-17T04:56:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: osdmap e90: 12 total, 12 up, 12 in 2024-05-17T04:56:11.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:11 smithi110 ceph-mon[35497]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-17T04:56:12.516 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: pgmap v1036: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:12.516 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6834/940026799,v1:172.21.15.105:6837/940026799] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6835/4268496386,v1:172.21.15.105:6838/4268496386] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.105:6836/4013930471,v1:172.21.15.105:6839/4013930471] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6834/3570871967,v1:172.21.15.110:6837/3570871967] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6835/2029600987,v1:172.21.15.110:6838/2029600987] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: mds.? [v2:172.21.15.110:6836/2491432349,v1:172.21.15.110:6839/2491432349] up:boot 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:56:12.517 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:12 smithi050 ceph-mon[31739]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:56:12.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: pgmap v1036: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6834/940026799,v1:172.21.15.105:6837/940026799] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6835/4268496386,v1:172.21.15.105:6838/4268496386] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.105:6836/4013930471,v1:172.21.15.105:6839/4013930471] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6834/3570871967,v1:172.21.15.110:6837/3570871967] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6835/2029600987,v1:172.21.15.110:6838/2029600987] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: mds.? [v2:172.21.15.110:6836/2491432349,v1:172.21.15.110:6839/2491432349] up:boot 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:56:12.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:12 smithi105 ceph-mon[34795]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: pgmap v1036: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6834/940026799,v1:172.21.15.105:6837/940026799] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6835/4268496386,v1:172.21.15.105:6838/4268496386] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.105:6836/4013930471,v1:172.21.15.105:6839/4013930471] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6834/3570871967,v1:172.21.15.110:6837/3570871967] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6835/2029600987,v1:172.21.15.110:6838/2029600987] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: mds.? [v2:172.21.15.110:6836/2491432349,v1:172.21.15.110:6839/2491432349] up:boot 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-17T04:56:12.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-17T04:56:12.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-17T04:56:12.568 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:12 smithi110 ceph-mon[35497]: from='mgr.14152 172.21.15.50:0/2371328411' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-17T04:56:13.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 -- 172.21.15.50:0/223082688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c102980 msgr2=0x7f737c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 --2- 172.21.15.50:0/223082688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c102980 0x7f737c102d90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7370009990 tx=0x7f7370009ca0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 -- 172.21.15.50:0/223082688 shutdown_connections 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 --2- 172.21.15.50:0/223082688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1032d0 0x7f737c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 --2- 172.21.15.50:0/223082688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c1089b0 0x7f737c108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 --2- 172.21.15.50:0/223082688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c102980 0x7f737c102d90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 -- 172.21.15.50:0/223082688 >> 172.21.15.50:0/223082688 conn(0x7f737c0fe6b0 msgr2=0x7f737c100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 -- 172.21.15.50:0/223082688 shutdown_connections 2024-05-17T04:56:13.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.267+0000 7f737bfff700 1 -- 172.21.15.50:0/223082688 wait complete. 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.268+0000 7f737bfff700 1 Processor -- start 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.268+0000 7f737bfff700 1 -- start start 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c102980 0x7f737c0feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1089b0 0x7f737c0ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:13.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f737c1129f0 con 0x7f737c1032d0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f737c112870 con 0x7f737c102980 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737bfff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f737c112b70 con 0x7f737c1089b0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42070/0 (socket says 172.21.15.50:42070) 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 -- 172.21.15.50:0/3787945480 learned_addr learned my addr 172.21.15.50:0/3787945480 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737b7fe700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1089b0 0x7f737c0ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737affd700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c102980 0x7f737c0feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 -- 172.21.15.50:0/3787945480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1089b0 msgr2=0x7f737c0ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1089b0 0x7f737c0ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 -- 172.21.15.50:0/3787945480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c102980 msgr2=0x7f737c0feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c102980 0x7f737c0feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 -- 172.21.15.50:0/3787945480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c1a87f0 con 0x7f737c1032d0 2024-05-17T04:56:13.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.269+0000 7f737a7fc700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7370000c00 tx=0x7f73700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.270+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370027070 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.270+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f737000fc00 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.270+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370022cd0 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.270+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c1a8a80 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.270+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c1a8f30 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.271+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c102d90 con 0x7f737c1032d0 2024-05-17T04:56:13.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.273+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f737002a070 con 0x7f737c1032d0 2024-05-17T04:56:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.273+0000 7f7363fff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 0x7f73640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.273+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f73700e37d0 con 0x7f737c1032d0 2024-05-17T04:56:13.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.273+0000 7f737affd700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 0x7f73640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:13.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.274+0000 7f737affd700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 0x7f73640b4e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:13.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.276+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f737005fdf0 con 0x7f737c1032d0 2024-05-17T04:56:13.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:13.542+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f737c1000c0 con 0x7f737c1032d0 2024-05-17T04:56:14.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.246+0000 7f7363fff700 1 -- 172.21.15.50:0/3787945480 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f7370013070 con 0x7f737c1032d0 2024-05-17T04:56:14.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 msgr2=0x7f73640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 0x7f73640b4e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c005e50 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 msgr2=0x7f737c0ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7370000c00 tx=0x7f73700057c0 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 shutdown_connections 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f737c1089b0 0x7f737c0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f737c102980 0x7f737c0feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f73640b2900 0x7f73640b4e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 --2- 172.21.15.50:0/3787945480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f737c1032d0 0x7f737c0ff0d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 >> 172.21.15.50:0/3787945480 conn(0x7f737c0fe6b0 msgr2=0x7f737c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 shutdown_connections 2024-05-17T04:56:14.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:14.250+0000 7f737bfff700 1 -- 172.21.15.50:0/3787945480 wait complete. 2024-05-17T04:56:14.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:14 smithi105 ceph-mon[34795]: pgmap v1037: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:14.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:14 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:56:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:14 smithi110 ceph-mon[35497]: pgmap v1037: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:14.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:14 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:56:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:14 smithi050 ceph-mon[31739]: pgmap v1037: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 614 B/s wr, 6 op/s 2024-05-17T04:56:14.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:14 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-17T04:56:15.120 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T04:56:15.377 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:15.500 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:15 smithi050 ceph-mon[31739]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T04:56:15.500 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:15 smithi050 ceph-mon[31739]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T04:56:15.500 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:15 smithi050 ceph-mon[31739]: Cluster is now healthy 2024-05-17T04:56:15.500 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:15 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:56:15.500 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:15 smithi050 ceph-mon[31739]: fsmap 12 up:standby 2024-05-17T04:56:15.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:15 smithi105 ceph-mon[34795]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T04:56:15.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:15 smithi105 ceph-mon[34795]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T04:56:15.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:15 smithi105 ceph-mon[34795]: Cluster is now healthy 2024-05-17T04:56:15.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:15 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:56:15.565 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:15 smithi105 ceph-mon[34795]: fsmap 12 up:standby 2024-05-17T04:56:15.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:15 smithi110 ceph-mon[35497]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T04:56:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:15 smithi110 ceph-mon[35497]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-17T04:56:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:15 smithi110 ceph-mon[35497]: Cluster is now healthy 2024-05-17T04:56:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:15 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3787945480' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-17T04:56:15.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:15 smithi110 ceph-mon[35497]: fsmap 12 up:standby 2024-05-17T04:56:16.536 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:16 smithi050 ceph-mon[31739]: pgmap v1038: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T04:56:16.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:16 smithi105 ceph-mon[34795]: pgmap v1038: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T04:56:16.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:16 smithi110 ceph-mon[35497]: pgmap v1038: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 -- 172.21.15.50:0/1000951995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 msgr2=0x7f9c4410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 --2- 172.21.15.50:0/1000951995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c4410f9e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b0a0 tx=0x7f9c3c00b3b0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 -- 172.21.15.50:0/1000951995 shutdown_connections 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 --2- 172.21.15.50:0/1000951995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c4410f9e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 --2- 172.21.15.50:0/1000951995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44102e00 0x7f9c44103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 --2- 172.21.15.50:0/1000951995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44107730 0x7f9c44107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 -- 172.21.15.50:0/1000951995 >> 172.21.15.50:0/1000951995 conn(0x7f9c44079bd0 msgr2=0x7f9c44079fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 -- 172.21.15.50:0/1000951995 shutdown_connections 2024-05-17T04:56:17.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.265+0000 7f9c4895c700 1 -- 172.21.15.50:0/1000951995 wait complete. 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 Processor -- start 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 -- start start 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44102e00 0x7f9c440ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44107730 0x7f9c440fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:17.269 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112b70 con 0x7f9c44108080 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441129f0 con 0x7f9c44107730 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.266+0000 7f9c4895c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112cf0 con 0x7f9c44102e00 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42086/0 (socket says 172.21.15.50:42086) 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 -- 172.21.15.50:0/3068708518 learned_addr learned my addr 172.21.15.50:0/3068708518 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c4259c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44102e00 0x7f9c440ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:17.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c41d9b700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44107730 0x7f9c440fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 -- 172.21.15.50:0/3068708518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44102e00 msgr2=0x7f9c440ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44102e00 0x7f9c440ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 -- 172.21.15.50:0/3068708518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44107730 msgr2=0x7f9c440fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44107730 0x7f9c440fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 -- 172.21.15.50:0/3068708518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441a8990 con 0x7f9c44108080 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c42d9d700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b830 tx=0x7f9c3c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c021070 con 0x7f9c44108080 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c3c007b80 con 0x7f9c44108080 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441a8c20 con 0x7f9c44108080 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0052a0 con 0x7f9c44108080 2024-05-17T04:56:17.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.267+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441a9070 con 0x7f9c44108080 2024-05-17T04:56:17.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.269+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c44103210 con 0x7f9c44108080 2024-05-17T04:56:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.270+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c3c024070 con 0x7f9c44108080 2024-05-17T04:56:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.271+0000 7f9c337fe700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 0x7f9c2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.272+0000 7f9c4259c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 0x7f9c2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.272+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f9c3c00f020 con 0x7f9c44108080 2024-05-17T04:56:17.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.272+0000 7f9c4259c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 0x7f9c2c0b4d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400ac50 tx=0x7f9c3400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:17.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.275+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c3c02b020 con 0x7f9c44108080 2024-05-17T04:56:17.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:17.490+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c44064440 con 0x7f9c44108080 2024-05-17T04:56:18.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.268+0000 7f9c337fe700 1 -- 172.21.15.50:0/3068708518 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f9c3c0aba40 con 0x7f9c44108080 2024-05-17T04:56:18.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.272+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 msgr2=0x7f9c2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:18.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.272+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 0x7f9c2c0b4d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400ac50 tx=0x7f9c3400a380 comp rx=0 tx=0).stop 2024-05-17T04:56:18.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.272+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 msgr2=0x7f9c44100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.272+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b830 tx=0x7f9c3c004d00 comp rx=0 tx=0).stop 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 shutdown_connections 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7f9c2c0b2880 0x7f9c2c0b4d90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c44108080 0x7f9c44100490 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9c44102e00 0x7f9c440ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 --2- 172.21.15.50:0/3068708518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c44107730 0x7f9c440fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:18.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 >> 172.21.15.50:0/3068708518 conn(0x7f9c44079bd0 msgr2=0x7f9c4406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:18.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.273+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 shutdown_connections 2024-05-17T04:56:18.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:18.274+0000 7f9c4895c700 1 -- 172.21.15.50:0/3068708518 wait complete. 2024-05-17T04:56:18.275 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_metadata' removed 2024-05-17T04:56:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:18 smithi105 ceph-mon[34795]: pgmap v1039: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-17T04:56:18.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:18 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:18 smithi110 ceph-mon[35497]: pgmap v1039: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-17T04:56:18.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:18 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:18 smithi050 ceph-mon[31739]: pgmap v1039: 129 pgs: 129 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-17T04:56:18.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:18 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:18.963 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T04:56:19.219 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:19.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:19 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:19.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:19 smithi105 ceph-mon[34795]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T04:56:19.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:19 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:19.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:19 smithi110 ceph-mon[35497]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T04:56:19.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:19 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/3068708518' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:19.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:19 smithi050 ceph-mon[31739]: osdmap e91: 12 total, 12 up, 12 in 2024-05-17T04:56:20.413 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:20 smithi050 ceph-mon[31739]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:20.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:20 smithi105 ceph-mon[34795]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:20.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:20 smithi110 ceph-mon[35497]: pgmap v1041: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 -- 172.21.15.50:0/1126673942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600104e10 msgr2=0x7fd600105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 --2- 172.21.15.50:0/1126673942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600104e10 0x7fd600105220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000b0a0 tx=0x7fd5f000b3b0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 -- 172.21.15.50:0/1126673942 shutdown_connections 2024-05-17T04:56:21.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 --2- 172.21.15.50:0/1126673942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 0x7fd600110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 --2- 172.21.15.50:0/1126673942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600104e10 0x7fd600105220 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 --2- 172.21.15.50:0/1126673942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600105760 0x7fd60010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 -- 172.21.15.50:0/1126673942 >> 172.21.15.50:0/1126673942 conn(0x7fd600079b40 msgr2=0x7fd600079f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 -- 172.21.15.50:0/1126673942 shutdown_connections 2024-05-17T04:56:21.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.001+0000 7fd604f5e700 1 -- 172.21.15.50:0/1126673942 wait complete. 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.002+0000 7fd604f5e700 1 Processor -- start 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.002+0000 7fd604f5e700 1 -- start start 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600104e10 0x7fd600102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 0x7fd600103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:21.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113c70 con 0x7fd600105760 2024-05-17T04:56:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113af0 con 0x7fd60010e240 2024-05-17T04:56:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd604f5e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd600113df0 con 0x7fd600104e10 2024-05-17T04:56:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40754/0 (socket says 172.21.15.50:40754) 2024-05-17T04:56:21.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 -- 172.21.15.50:0/463740948 learned_addr learned my addr 172.21.15.50:0/463740948 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fe59c700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600104e10 0x7fd600102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fed9d700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 0x7fd600103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 -- 172.21.15.50:0/463740948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600104e10 msgr2=0x7fd600102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600104e10 0x7fd600102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 -- 172.21.15.50:0/463740948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 msgr2=0x7fd600103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:21.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 0x7fd600103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 -- 172.21.15.50:0/463740948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6001a8a20 con 0x7fd600105760 2024-05-17T04:56:21.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.003+0000 7fd5fdd9b700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d950 tx=0x7fd5f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:21.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.004+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4016070 con 0x7fd600105760 2024-05-17T04:56:21.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.004+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6001a8cb0 con 0x7fd600105760 2024-05-17T04:56:21.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.004+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f4009dd0 con 0x7fd600105760 2024-05-17T04:56:21.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.004+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6001a9190 con 0x7fd600105760 2024-05-17T04:56:21.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.004+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400f3c0 con 0x7fd600105760 2024-05-17T04:56:21.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.005+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60006c8c0 con 0x7fd600105760 2024-05-17T04:56:21.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.006+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd5f4019070 con 0x7fd600105760 2024-05-17T04:56:21.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.007+0000 7fd5ef7fe700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 0x7fd5e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:21.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.007+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8854+0+0 (secure 0 0 0) 0x7fd5f40e25f0 con 0x7fd600105760 2024-05-17T04:56:21.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.007+0000 7fd5fe59c700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 0x7fd5e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:21.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.008+0000 7fd5fe59c700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 0x7fd5e80b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:21.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.010+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5f40aa040 con 0x7fd600105760 2024-05-17T04:56:21.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.226+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd600104030 con 0x7fd600105760 2024-05-17T04:56:21.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.281+0000 7fd5ef7fe700 1 -- 172.21.15.50:0/463740948 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fd5f40a9940 con 0x7fd600105760 2024-05-17T04:56:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 msgr2=0x7fd5e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:21.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 0x7fd5e80b4d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009900 tx=0x7fd5f000b7b0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 msgr2=0x7fd600103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d950 tx=0x7fd5f400dc60 comp rx=0 tx=0).stop 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 shutdown_connections 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.284+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd60010e240 0x7fd600103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600104e10 0x7fd600102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fd5e80b2880 0x7fd5e80b4d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:21.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 --2- 172.21.15.50:0/463740948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd600105760 0x7fd600103040 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400d950 tx=0x7fd5f400dc60 comp rx=0 tx=0).stop 2024-05-17T04:56:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 >> 172.21.15.50:0/463740948 conn(0x7fd600079b40 msgr2=0x7fd6000ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 shutdown_connections 2024-05-17T04:56:21.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:21.285+0000 7fd604f5e700 1 -- 172.21.15.50:0/463740948 wait complete. 2024-05-17T04:56:21.287 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data' removed 2024-05-17T04:56:21.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:21 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:21.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:21 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:21.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:21 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:21.952 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc shell --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-17T04:56:22.224 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/mon.a/config 2024-05-17T04:56:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:22 smithi105 ceph-mon[34795]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:22 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:22.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:22 smithi105 ceph-mon[34795]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T04:56:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:22 smithi110 ceph-mon[35497]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:22.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:22 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:22.567 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:22 smithi110 ceph-mon[35497]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T04:56:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:22 smithi050 ceph-mon[31739]: pgmap v1042: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:22 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/463740948' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:22.575 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:22 smithi050 ceph-mon[31739]: osdmap e92: 12 total, 12 up, 12 in 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 -- 172.21.15.50:0/1979261397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 msgr2=0x7fed14107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 --2- 172.21.15.50:0/1979261397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed14107b60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009990 tx=0x7fecfc009ca0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 -- 172.21.15.50:0/1979261397 shutdown_connections 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 --2- 172.21.15.50:0/1979261397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed141080a0 0x7fed1410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 --2- 172.21.15.50:0/1979261397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed14102e70 0x7fed14103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 --2- 172.21.15.50:0/1979261397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed14107b60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 -- 172.21.15.50:0/1979261397 >> 172.21.15.50:0/1979261397 conn(0x7fed14079ae0 msgr2=0x7fed14079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.054+0000 7fed1a67a700 1 -- 172.21.15.50:0/1979261397 shutdown_connections 2024-05-17T04:56:24.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.055+0000 7fed1a67a700 1 -- 172.21.15.50:0/1979261397 wait complete. 2024-05-17T04:56:24.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.055+0000 7fed1a67a700 1 Processor -- start 2024-05-17T04:56:24.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.055+0000 7fed1a67a700 1 -- start start 2024-05-17T04:56:24.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.055+0000 7fed1a67a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14102e70 0x7fed1419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.055+0000 7fed1a67a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed1a67a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 0x7fed1419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed1a67a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed14112b60 con 0x7fed14107750 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed1a67a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed141129e0 con 0x7fed141080a0 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed1a67a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed14112ce0 con 0x7fed14102e70 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:40768/0 (socket says 172.21.15.50:40768) 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 -- 172.21.15.50:0/21264528 learned_addr learned my addr 172.21.15.50:0/21264528 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed18c17700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 0x7fed1419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed13fff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14102e70 0x7fed1419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 -- 172.21.15.50:0/21264528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14102e70 msgr2=0x7fed1419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14102e70 0x7fed1419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 -- 172.21.15.50:0/21264528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 msgr2=0x7fed1419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 0x7fed1419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 -- 172.21.15.50:0/21264528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1419f490 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed18c17700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 0x7fed1419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed0bfff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc027070 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1419f720 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.056+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecfc00fdb0 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.057+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0173f0 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.057+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1419fc50 con 0x7fed14107750 2024-05-17T04:56:24.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.058+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14103280 con 0x7fed14107750 2024-05-17T04:56:24.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.060+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fecfc02a070 con 0x7fed14107750 2024-05-17T04:56:24.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.061+0000 7fed11ffb700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 0x7fed000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T04:56:24.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.061+0000 7fed13fff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 0x7fed000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T04:56:24.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.061+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8324+0+0 (secure 0 0 0) 0x7fecfc0e37f0 con 0x7fed14107750 2024-05-17T04:56:24.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.062+0000 7fed13fff700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 0x7fed000b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed04005ea0 tx=0x7fed04005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T04:56:24.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.064+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecfc031080 con 0x7fed14107750 2024-05-17T04:56:24.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:24.280+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed1419b9d0 con 0x7fed14107750 2024-05-17T04:56:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:24 smithi105 ceph-mon[34795]: pgmap v1044: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:24.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:24 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:24.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:24 smithi110 ceph-mon[35497]: pgmap v1044: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:24.570 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:24 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:24 smithi050 ceph-mon[31739]: pgmap v1044: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:24.576 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:24 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-17T04:56:25.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.234+0000 7fed11ffb700 1 -- 172.21.15.50:0/21264528 <== mon.0 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) ==== 178+0+0 (secure 0 0 0) 0x7fecfc013070 con 0x7fed14107750 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 msgr2=0x7fed000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 0x7fed000b4d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed04005ea0 tx=0x7fed04005e30 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 msgr2=0x7fed1419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc0047b0 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 shutdown_connections 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed141080a0 0x7fed1419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed14102e70 0x7fed1419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:6800/1662041432,v1:172.21.15.50:6801/1662041432] conn(0x7fed000b2880 0x7fed000b4d90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 --2- 172.21.15.50:0/21264528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed14107750 0x7fed1419ac10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T04:56:25.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.239+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 >> 172.21.15.50:0/21264528 conn(0x7fed14079ae0 msgr2=0x7fed1406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T04:56:25.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.240+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 shutdown_connections 2024-05-17T04:56:25.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-17T04:56:25.240+0000 7fed1a67a700 1 -- 172.21.15.50:0/21264528 wait complete. 2024-05-17T04:56:25.243 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'cephfs_data_ec' removed 2024-05-17T04:56:26.042 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:56:26.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:56:26.109 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:56:26.147 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-17T04:56:26.148 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T04:56:26.169 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T04:56:26.184 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-17T04:56:26.204 INFO:tasks.cephadm:Stopping all daemons... 2024-05-17T04:56:26.204 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-17T04:56:26.204 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a 2024-05-17T04:56:26.253 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 ceph-mon[31739]: pgmap v1045: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:26.253 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 ceph-mon[31739]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:26.253 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 ceph-mon[31739]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T04:56:26.557 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 systemd[1]: Stopping Ceph mon.a for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:26 smithi105 ceph-mon[34795]: pgmap v1045: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:26 smithi105 ceph-mon[34795]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:26.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:26 smithi105 ceph-mon[34795]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T04:56:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:26 smithi110 ceph-mon[35497]: pgmap v1045: 33 pgs: 33 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-17T04:56:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:26 smithi110 ceph-mon[35497]: from='client.? 172.21.15.50:0/21264528' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-17T04:56:26.566 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:26 smithi110 ceph-mon[35497]: osdmap e93: 12 total, 12 up, 12 in 2024-05-17T04:56:26.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a[31735]: 2024-05-17T04:56:26.556+0000 7fc492d67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:26.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a[31735]: 2024-05-17T04:56:26.556+0000 7fc492d67700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-17T04:56:26.826 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:26 smithi050 podman[99506]: 2024-05-17 04:56:26.742217949 +0000 UTC m=+0.322395018 container died ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:56:27.325 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:27 smithi050 podman[99506]: 2024-05-17 04:56:27.11658848 +0000 UTC m=+0.696765526 container cleanup ae0dc175e3a33fdd3954facbe60397565179b8638663985ba8c31ce45f14d2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-17T04:56:27.326 INFO:journalctl@ceph.mon.a.smithi050.stdout:May 17 04:56:27 smithi050 bash[99506]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-a 2024-05-17T04:56:27.592 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-17T04:56:27.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:27.709 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-17T04:56:27.709 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-17T04:56:27.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.b 2024-05-17T04:56:28.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:27 smithi105 systemd[1]: Stopping Ceph mon.b for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-b[34790]: 2024-05-17T04:56:28.150+0000 7fd6cc28a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-b[34790]: 2024-05-17T04:56:28.150+0000 7fd6cc28a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-17T04:56:28.564 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:28 smithi105 podman[72576]: 2024-05-17 04:56:28.273212811 +0000 UTC m=+0.247117858 container died f78e990643ecc43c2936be4b2d88943dceb1a863ac3c839af4f46bf5f2020a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-17T04:56:29.064 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:28 smithi105 podman[72576]: 2024-05-17 04:56:28.826423531 +0000 UTC m=+0.800328578 container cleanup f78e990643ecc43c2936be4b2d88943dceb1a863ac3c839af4f46bf5f2020a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:56:29.065 INFO:journalctl@ceph.mon.b.smithi105.stdout:May 17 04:56:28 smithi105 bash[72576]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-b 2024-05-17T04:56:29.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-17T04:56:29.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:29.310 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-17T04:56:29.310 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-17T04:56:29.310 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.c 2024-05-17T04:56:29.655 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:29 smithi110 systemd[1]: Stopping Ceph mon.c for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:29.911 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c[35493]: 2024-05-17T04:56:29.655+0000 7f2a26983700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:29.911 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c[35493]: 2024-05-17T04:56:29.655+0000 7f2a26983700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-17T04:56:30.316 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:29 smithi110 podman[73348]: 2024-05-17 04:56:29.911259399 +0000 UTC m=+0.367929276 container died 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-17T04:56:31.274 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:30 smithi110 podman[73348]: 2024-05-17 04:56:30.955928355 +0000 UTC m=+1.412598237 container cleanup 0271de19e04b9258daad16eeeab3f8a0c62ce44f404315517316bdd8191f2639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-17T04:56:31.274 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 17 04:56:30 smithi110 bash[73348]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mon-c 2024-05-17T04:56:31.440 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-17T04:56:31.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:31.491 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-17T04:56:31.491 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-17T04:56:31.491 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x 2024-05-17T04:56:31.822 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:31 smithi050 systemd[1]: Stopping Ceph mgr.x for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:32.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x[31963]: 2024-05-17T04:56:31.821+0000 7fe476ec3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:32.076 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:31 smithi050 podman[99628]: 2024-05-17 04:56:31.853663737 +0000 UTC m=+0.159934537 container died 5b7105165780138bc00349df9629873575e353521f012af119cba9151fbc6070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-17T04:56:32.449 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 podman[99628]: 2024-05-17 04:56:32.198660744 +0000 UTC m=+0.504931536 container cleanup 5b7105165780138bc00349df9629873575e353521f012af119cba9151fbc6070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-17T04:56:32.450 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 bash[99628]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x 2024-05-17T04:56:32.684 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-17T04:56:32.709 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 podman[99640]: 2024-05-17 04:56:32.450758626 +0000 UTC m=+0.582151591 container remove 5b7105165780138bc00349df9629873575e353521f012af119cba9151fbc6070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:931809a9e5d295bd61984693cac44fe131ded4bc, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-17T04:56:32.709 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service: Deactivated successfully. 2024-05-17T04:56:32.709 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 systemd[1]: Stopped Ceph mgr.x for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:56:32.709 INFO:journalctl@ceph.mgr.x.smithi050.stdout:May 17 04:56:32 smithi050 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.x.service: Consumed 1min 39.127s CPU time. 2024-05-17T04:56:32.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:32.723 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-17T04:56:32.723 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-17T04:56:32.723 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.y 2024-05-17T04:56:33.064 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:56:32 smithi105 systemd[1]: Stopping Ceph mgr.y for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:33.449 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:56:33 smithi105 podman[72701]: 2024-05-17 04:56:33.14637493 +0000 UTC m=+0.218831950 container died 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2) 2024-05-17T04:56:33.800 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:56:33 smithi105 podman[72701]: 2024-05-17 04:56:33.541173682 +0000 UTC m=+0.613630680 container cleanup 2b4c477648d0e0b2aa5be4c728fbad0320848685c9b19b7dbc92db476c8b4d39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:56:33.800 INFO:journalctl@ceph.mgr.y.smithi105.stdout:May 17 04:56:33 smithi105 bash[72701]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-y 2024-05-17T04:56:33.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-17T04:56:33.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:33.963 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-17T04:56:33.964 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-17T04:56:33.964 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.z 2024-05-17T04:56:34.317 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:56:34 smithi110 systemd[1]: Stopping Ceph mgr.z for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:34.575 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:56:34 smithi110 podman[73475]: 2024-05-17 04:56:34.32049394 +0000 UTC m=+0.159736954 container died 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:56:35.483 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:56:35 smithi110 podman[73475]: 2024-05-17 04:56:35.223767431 +0000 UTC m=+1.063010437 container cleanup 169a9b32a072dd85d6e59dda93634f1cc946b002d408281b9c8fb653d7c36c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T04:56:35.483 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 17 04:56:35 smithi110 bash[73475]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mgr-z 2024-05-17T04:56:35.650 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-17T04:56:35.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:35.687 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-17T04:56:35.687 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-17T04:56:35.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.0 2024-05-17T04:56:36.003 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:35 smithi050 systemd[1]: Stopping Ceph osd.0 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:36.326 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0[44169]: 2024-05-17T04:56:36.003+0000 7fbe0809d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:36.326 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0[44169]: 2024-05-17T04:56:36.003+0000 7fbe0809d700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-17T04:56:36.326 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0[44169]: 2024-05-17T04:56:36.003+0000 7fbe0809d700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:56:41.746 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:41 smithi050 podman[99749]: 2024-05-17 04:56:41.460937152 +0000 UTC m=+5.569134878 container died 1bb80c3b0d97d0de1e204128547650129367787cd936f95edd23194876989369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-17T04:56:42.076 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:41 smithi050 podman[99749]: 2024-05-17 04:56:41.837656489 +0000 UTC m=+5.945854215 container cleanup 1bb80c3b0d97d0de1e204128547650129367787cd936f95edd23194876989369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-17T04:56:42.076 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:41 smithi050 bash[99749]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0 2024-05-17T04:56:42.473 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99766]: 2024-05-17 04:56:42.147858527 +0000 UTC m=+0.689799359 container remove 1bb80c3b0d97d0de1e204128547650129367787cd936f95edd23194876989369 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-05-17T04:56:42.759 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99859]: 2024-05-17 04:56:42.377184697 +0000 UTC m=+0.018271422 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:56:42.760 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99859]: 2024-05-17 04:56:42.512668989 +0000 UTC m=+0.153755723 container create 55ffca3a46f411f1cbc20758fdacb89a33aa8fffb538db4ff53c3f94a5e7bc4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:56:42.760 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99859]: 2024-05-17 04:56:42.760759899 +0000 UTC m=+0.401846619 container init 55ffca3a46f411f1cbc20758fdacb89a33aa8fffb538db4ff53c3f94a5e7bc4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514) 2024-05-17T04:56:43.076 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99859]: 2024-05-17 04:56:42.764312534 +0000 UTC m=+0.405399252 container start 55ffca3a46f411f1cbc20758fdacb89a33aa8fffb538db4ff53c3f94a5e7bc4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-17T04:56:43.076 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:42 smithi050 podman[99859]: 2024-05-17 04:56:42.860754205 +0000 UTC m=+0.501840936 container attach 55ffca3a46f411f1cbc20758fdacb89a33aa8fffb538db4ff53c3f94a5e7bc4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD) 2024-05-17T04:56:43.656 INFO:journalctl@ceph.osd.0.smithi050.stdout:May 17 04:56:43 smithi050 podman[99859]: 2024-05-17 04:56:43.350048049 +0000 UTC m=+0.991134766 container died 55ffca3a46f411f1cbc20758fdacb89a33aa8fffb538db4ff53c3f94a5e7bc4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:56:44.014 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-17T04:56:44.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:44.060 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-17T04:56:44.060 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-17T04:56:44.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.1 2024-05-17T04:56:44.388 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:44 smithi105 systemd[1]: Stopping Ceph osd.1 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:44.814 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1[39370]: 2024-05-17T04:56:44.387+0000 7f445e270700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:44.814 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1[39370]: 2024-05-17T04:56:44.387+0000 7f445e270700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-17T04:56:44.815 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1[39370]: 2024-05-17T04:56:44.387+0000 7f445e270700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:56:50.239 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:49 smithi105 podman[72839]: 2024-05-17 04:56:49.949323284 +0000 UTC m=+5.688866882 container died 6c0baee0344aa2953390f4dd2b98fca6a87d14eb98ebf863c1bd0e0b2abce643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:56:50.564 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:50 smithi105 podman[72839]: 2024-05-17 04:56:50.338538394 +0000 UTC m=+6.078081986 container cleanup 6c0baee0344aa2953390f4dd2b98fca6a87d14eb98ebf863c1bd0e0b2abce643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:56:50.564 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:50 smithi105 bash[72839]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1 2024-05-17T04:56:51.020 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:50 smithi105 podman[72855]: 2024-05-17 04:56:50.640250707 +0000 UTC m=+0.685412917 container remove 6c0baee0344aa2953390f4dd2b98fca6a87d14eb98ebf863c1bd0e0b2abce643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:56:51.285 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:50.923746045 +0000 UTC m=+0.029744172 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:56:51.285 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:51.032930896 +0000 UTC m=+0.138929006 container create 9de63fd0ea4b6e042c023d73839cbf992e79ffd29631bb2291723cd6ecad6420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:56:51.564 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:51.285924621 +0000 UTC m=+0.391922739 container init 9de63fd0ea4b6e042c023d73839cbf992e79ffd29631bb2291723cd6ecad6420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:56:51.564 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:51.289312706 +0000 UTC m=+0.395310832 container start 9de63fd0ea4b6e042c023d73839cbf992e79ffd29631bb2291723cd6ecad6420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-17T04:56:51.565 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:51.402748071 +0000 UTC m=+0.508746209 container attach 9de63fd0ea4b6e042c023d73839cbf992e79ffd29631bb2291723cd6ecad6420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-17T04:56:52.064 INFO:journalctl@ceph.osd.1.smithi105.stdout:May 17 04:56:51 smithi105 podman[72948]: 2024-05-17 04:56:51.791014277 +0000 UTC m=+0.897012396 container died 9de63fd0ea4b6e042c023d73839cbf992e79ffd29631bb2291723cd6ecad6420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-17T04:56:52.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-17T04:56:52.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:56:52.483 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-17T04:56:52.483 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-17T04:56:52.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.2 2024-05-17T04:56:52.770 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:52 smithi110 systemd[1]: Stopping Ceph osd.2 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:56:53.067 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2[40638]: 2024-05-17T04:56:52.769+0000 7f46d8a68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:56:53.067 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2[40638]: 2024-05-17T04:56:52.769+0000 7f46d8a68700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-17T04:56:53.067 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2[40638]: 2024-05-17T04:56:52.769+0000 7f46d8a68700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:56:58.562 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:58 smithi110 podman[73608]: 2024-05-17 04:56:58.303942239 +0000 UTC m=+5.628325567 container died d0dd6f2dee21cb7258b9c298389f8fe63dea6117d5c5ccd459b0a1803889206e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:56:58.816 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:58 smithi110 podman[73608]: 2024-05-17 04:56:58.620069725 +0000 UTC m=+5.944453044 container cleanup d0dd6f2dee21cb7258b9c298389f8fe63dea6117d5c5ccd459b0a1803889206e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-05-17T04:56:58.817 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:58 smithi110 bash[73608]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2 2024-05-17T04:56:59.222 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:58 smithi110 podman[73624]: 2024-05-17 04:56:58.908936891 +0000 UTC m=+0.598789101 container remove d0dd6f2dee21cb7258b9c298389f8fe63dea6117d5c5ccd459b0a1803889206e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T04:56:59.479 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:59 smithi110 podman[73713]: 2024-05-17 04:56:59.126099834 +0000 UTC m=+0.016869511 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:56:59.479 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:59 smithi110 podman[73713]: 2024-05-17 04:56:59.251891514 +0000 UTC m=+0.142661254 container create 1189315638a2029d7c61128cf870bdfe1223fae2e92acfb901c8eecf118e760d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-05-17T04:56:59.479 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:59 smithi110 podman[73713]: 2024-05-17 04:56:59.479699229 +0000 UTC m=+0.370468910 container init 1189315638a2029d7c61128cf870bdfe1223fae2e92acfb901c8eecf118e760d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:56:59.817 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:59 smithi110 podman[73713]: 2024-05-17 04:56:59.483055873 +0000 UTC m=+0.373825553 container start 1189315638a2029d7c61128cf870bdfe1223fae2e92acfb901c8eecf118e760d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-17T04:56:59.817 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:56:59 smithi110 podman[73713]: 2024-05-17 04:56:59.588165394 +0000 UTC m=+0.478935084 container attach 1189315638a2029d7c61128cf870bdfe1223fae2e92acfb901c8eecf118e760d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T04:57:00.317 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 17 04:57:00 smithi110 podman[73713]: 2024-05-17 04:57:00.048412978 +0000 UTC m=+0.939182672 container died 1189315638a2029d7c61128cf870bdfe1223fae2e92acfb901c8eecf118e760d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:57:00.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-17T04:57:00.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:00.654 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-17T04:57:00.654 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-17T04:57:00.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.3 2024-05-17T04:57:00.976 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:00 smithi050 systemd[1]: Stopping Ceph osd.3 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:01.253 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:00 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:57:00.976+0000 7f24c9bd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:01.253 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:00 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:57:00.976+0000 7f24c9bd0700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-17T04:57:01.253 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:00 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:57:00.976+0000 7f24c9bd0700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:02.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:02 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:02.314+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:02.817 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:02.519+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:02.818 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:02.452+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:03.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:03.115+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:03.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:03 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:03.275+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:03.817 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:03.479+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:03.425+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:04.341 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:04 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:04.072+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:04.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:04 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:04.325+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:04.391+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:04.718 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:04.527+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:05.288 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:05 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:05.033+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:05.575 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:05 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3[50817]: 2024-05-17T04:57:05.524+0000 7f24c6186700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:40.047361+0000 front 2024-05-17T04:56:40.047416+0000 (oldest deadline 2024-05-17T04:57:05.347598+0000) 2024-05-17T04:57:05.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:05 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:05.308+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:05.818 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:05.376+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:05.818 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:05.570+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:06.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:06 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:06.002+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:06.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:06 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:06.279+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:06.576 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:06 smithi050 podman[99965]: 2024-05-17 04:57:06.467707589 +0000 UTC m=+5.610459806 container died c3582222ede8f71232d42d5918f2202dcb8e9a4417caa1ff68fd12d6a701fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-17T04:57:06.609 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:06.339+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:07.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:06.609+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:07.275 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:06 smithi050 podman[99965]: 2024-05-17 04:57:06.943949612 +0000 UTC m=+6.086701824 container cleanup c3582222ede8f71232d42d5918f2202dcb8e9a4417caa1ff68fd12d6a701fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-17T04:57:07.275 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:06 smithi050 bash[99965]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3 2024-05-17T04:57:07.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:07 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:07.013+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:07.575 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:07 smithi050 podman[99995]: 2024-05-17 04:57:07.412487518 +0000 UTC m=+0.943371213 container remove c3582222ede8f71232d42d5918f2202dcb8e9a4417caa1ff68fd12d6a701fe65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:57:07.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:07 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:07.273+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:07.816 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:07.630+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:07.382+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:08.066 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:07 smithi050 podman[100087]: 2024-05-17 04:57:07.647118055 +0000 UTC m=+0.018363338 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:08.066 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:07 smithi050 podman[100087]: 2024-05-17 04:57:07.788948844 +0000 UTC m=+0.160194160 container create 0e637afb650cfb93946e2f3c444e5a0ac51b39358b50971a215ee937579e55b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:08.066 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:08 smithi050 podman[100087]: 2024-05-17 04:57:08.066679691 +0000 UTC m=+0.437924961 container init 0e637afb650cfb93946e2f3c444e5a0ac51b39358b50971a215ee937579e55b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:57:08.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:08 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:08.007+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:08.319 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:08 smithi050 podman[100087]: 2024-05-17 04:57:08.07034046 +0000 UTC m=+0.441585748 container start 0e637afb650cfb93946e2f3c444e5a0ac51b39358b50971a215ee937579e55b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:08.319 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:08 smithi050 podman[100087]: 2024-05-17 04:57:08.175213066 +0000 UTC m=+0.546458351 container attach 0e637afb650cfb93946e2f3c444e5a0ac51b39358b50971a215ee937579e55b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T04:57:08.576 INFO:journalctl@ceph.osd.3.smithi050.stdout:May 17 04:57:08 smithi050 podman[100087]: 2024-05-17 04:57:08.387338585 +0000 UTC m=+0.758583866 container died 0e637afb650cfb93946e2f3c444e5a0ac51b39358b50971a215ee937579e55b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-17T04:57:08.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:08 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:08.318+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:08.817 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:08.650+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:08.408+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:09.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-17T04:57:09.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:09.231 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-17T04:57:09.231 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-17T04:57:09.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.4 2024-05-17T04:57:09.286 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:09 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:09.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:09.564 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:09 smithi105 systemd[1]: Stopping Ceph osd.4 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:09.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:09 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:09.340+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:09.697 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:09.395+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:10.032 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:09 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:57:09.568+0000 7f48ff474700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:10.032 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:09 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:57:09.568+0000 7f48ff474700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-17T04:57:10.032 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:09 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:57:09.568+0000 7f48ff474700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:10.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:09.697+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:10.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:10 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:10.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:10.723 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:10.350+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:10.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:10 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:10.355+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:11.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:10.723+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:11.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:11 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:11.066+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:11.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:11 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:11.066+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:11.566 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:11.382+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:11.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:11.227+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:11.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:11 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:11.312+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:12.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:11.689+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:12.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:12 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:12.104+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:12.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:12 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:12.104+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:12.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:12 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:12.260+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:12.566 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:12 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:12.402+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:12.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:12 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:12.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:13.064 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:12 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:57:12.687+0000 7f48fba2a700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.585075+0000 front 2024-05-17T04:56:47.585111+0000 (oldest deadline 2024-05-17T04:57:12.285237+0000) 2024-05-17T04:57:13.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:12 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:12.665+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:13.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:13 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:13.142+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:13.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:13 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:13.142+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:13.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:13 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:13.253+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:13.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:13 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:13.381+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:13.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:13.346+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:13.826 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:13 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:13.347+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:14.064 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:13 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4[45947]: 2024-05-17T04:57:13.710+0000 7f48fba2a700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.585075+0000 front 2024-05-17T04:56:47.585111+0000 (oldest deadline 2024-05-17T04:57:12.285237+0000) 2024-05-17T04:57:14.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:13 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:13.625+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:14.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:14 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:14.140+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:14.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:14 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:14.140+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:14.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:14 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:14.373+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:14.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:14 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:14.279+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:14.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:14 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:14.333+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:14.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:14 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:14.333+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:14.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:14.306+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:14.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:14.321+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:14.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:14.321+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:15.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:14 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:14.612+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:15.375 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:15 smithi105 podman[73078]: 2024-05-17 04:57:15.097875918 +0000 UTC m=+5.655057549 container died c321e5b5aa1190645076bc5b44c618b2a25ce12ed6f21b74519e338d6859e3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-17T04:57:15.375 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:15 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:15.185+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:15.375 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:15 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:15.185+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:15.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:15.264+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:15.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:15.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:15.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:15.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:15.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:15.336+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:15.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:15.321+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:15.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:15 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:15.321+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:15.770 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:15 smithi105 podman[73078]: 2024-05-17 04:57:15.519078577 +0000 UTC m=+6.076260200 container cleanup c321e5b5aa1190645076bc5b44c618b2a25ce12ed6f21b74519e338d6859e3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-17T04:57:15.770 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:15 smithi105 bash[73078]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4 2024-05-17T04:57:15.770 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:15 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:15.375+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:16.064 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:15 smithi105 podman[73096]: 2024-05-17 04:57:15.770640194 +0000 UTC m=+0.667833049 container remove c321e5b5aa1190645076bc5b44c618b2a25ce12ed6f21b74519e338d6859e3a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS) 2024-05-17T04:57:16.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:15.578+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:16.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:15 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:15.578+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:16.473 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:16 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:16.328+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:16.474 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:16 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:16.152+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:16.474 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:16 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:16.152+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:16.474 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.086632546 +0000 UTC m=+0.016959911 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:16.474 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.196481632 +0000 UTC m=+0.126809019 container create db5355a3780d346c81c5d5648af8d160e57fecfd111cede714f276509c91f3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-17T04:57:16.474 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.474760423 +0000 UTC m=+0.405087788 container init db5355a3780d346c81c5d5648af8d160e57fecfd111cede714f276509c91f3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T04:57:16.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:16 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:16.254+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:16.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:16 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:16.386+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:16.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:16 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:16.386+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:16.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:16.318+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:16.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:16.281+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:16.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:16 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:16.281+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:16.778 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.478271273 +0000 UTC m=+0.408598641 container start db5355a3780d346c81c5d5648af8d160e57fecfd111cede714f276509c91f3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-17T04:57:16.778 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.574775318 +0000 UTC m=+0.505102700 container attach db5355a3780d346c81c5d5648af8d160e57fecfd111cede714f276509c91f3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T04:57:17.064 INFO:journalctl@ceph.osd.4.smithi105.stdout:May 17 04:57:16 smithi105 podman[73184]: 2024-05-17 04:57:16.778854867 +0000 UTC m=+0.709182235 container died db5355a3780d346c81c5d5648af8d160e57fecfd111cede714f276509c91f3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-17T04:57:17.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:16 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:16.595+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:17.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:16 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:16.595+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:17.411 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:17 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:17.306+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:17.411 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:17 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:17.123+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:17.411 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:17 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:17.123+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:17.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-17T04:57:17.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:17.468 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-17T04:57:17.468 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-17T04:57:17.468 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.5 2024-05-17T04:57:17.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:17.285+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:17.567 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 systemd[1]: Stopping Ceph osd.5 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:17.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:17.341+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:17.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:17.341+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:17.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:17.309+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:17.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:17.233+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:17.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:17 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:17.233+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:18.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:17.610+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:18.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:17.610+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:18.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:17.786+0000 7f7830585700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:18.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:17.786+0000 7f7830585700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-17T04:57:18.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:17 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:17.786+0000 7f7830585700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:18.352 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:18 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:18.091+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:18.352 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:18 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:18.091+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:18.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:18.291+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:18.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:18.375+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:18.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:18.375+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:18.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:18 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:18.331+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:18.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:18 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:18.273+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:18.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:18 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:18.273+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:18.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:18 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:18.353+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:19.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:18.634+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:19.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:18 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:18.634+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:19.398 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:19 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:19.106+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:19.399 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:19 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:19.106+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:19.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:19.311+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:19.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:19.311+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:19.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:19.245+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:19.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:19.245+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:19.658 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:19.328+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:19.658 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:19.330+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:19.658 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:19.330+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:19.658 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:19.330+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:19.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:19 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:19.399+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:20.066 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:19.658+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:20.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:19 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:19.658+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:20.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:20 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:20.077+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:20.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:20 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:20.077+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:20.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:20.303+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:20.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:20.369+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:20.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:20.369+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:20.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:20.369+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:20.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:20 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:20.315+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:20.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:20 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:20.315+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:20.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:20 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:20.231+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:20.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:20 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:20.231+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:20.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:20 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:20.231+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:20.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:20 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:20.365+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:21.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:20.609+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:21.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:20.609+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:21.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:20 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:20.609+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.708096+0000 front 2024-05-17T04:56:57.708127+0000 (oldest deadline 2024-05-17T04:57:20.008264+0000) 2024-05-17T04:57:21.398 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:21 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:21.126+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:21.398 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:21 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:21.126+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:21.399 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:21 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:21.126+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:21.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:21.338+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:21.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:21.338+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:21.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:21.264+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:21.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:21.264+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:21.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:21 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:21.264+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:21.632 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:21.318+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:21.632 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:21.335+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:21.632 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:21.335+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:21.633 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:21.335+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:21.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:21 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:21.399+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:22.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:21.632+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:22.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:21.632+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:22.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:21 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:21.632+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.708096+0000 front 2024-05-17T04:56:57.708127+0000 (oldest deadline 2024-05-17T04:57:20.008264+0000) 2024-05-17T04:57:22.427 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:22 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:22.114+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:22.427 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:22 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:22.115+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:22.427 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:22 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:22.115+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:22.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:22.305+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:22.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:22.305+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:22.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:22.377+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:22.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:22.377+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:22.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:22.377+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:22.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:22 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:22.362+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:22.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:22 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:22.362+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:22.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:22 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:22.255+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:22.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:22 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:22.255+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:22.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:22 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:22.255+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:22.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:22 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:22.427+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:23.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:22.621+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.104086+0000 front 2024-05-17T04:56:36.104043+0000 (oldest deadline 2024-05-17T04:57:02.004097+0000) 2024-05-17T04:57:23.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:22.621+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.105342+0000 front 2024-05-17T04:56:46.105320+0000 (oldest deadline 2024-05-17T04:57:14.906114+0000) 2024-05-17T04:57:23.067 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:22 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5[47528]: 2024-05-17T04:57:22.621+0000 7f7828b33700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.708096+0000 front 2024-05-17T04:56:57.708127+0000 (oldest deadline 2024-05-17T04:57:20.008264+0000) 2024-05-17T04:57:23.409 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:23 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:23.127+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:23.409 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:23 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:23.127+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:23.409 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:23 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:23.127+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:23.566 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:23 smithi110 podman[73831]: 2024-05-17 04:57:23.346249833 +0000 UTC m=+5.679464758 container died 05c41517a1d60568a6b314134490bca3f76e4a52244835fa0a8076df2d5a686e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:23.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:23 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:23.313+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:23.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:23 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:23.313+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:23.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:23 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:23.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:23.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:23 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:23.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:23.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:23 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:23.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:23.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:23 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:23.365+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:23.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:23 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:23.365+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:23.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:23 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:23.223+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:23.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:23 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:23.223+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:23.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:23 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:23.223+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:23.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:23 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:23.410+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:23.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:23 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:23.410+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:24.029 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:23 smithi110 podman[73831]: 2024-05-17 04:57:23.737319503 +0000 UTC m=+6.070534431 container cleanup 05c41517a1d60568a6b314134490bca3f76e4a52244835fa0a8076df2d5a686e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:57:24.035 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:23 smithi110 bash[73831]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5 2024-05-17T04:57:24.319 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73846]: 2024-05-17 04:57:24.030487556 +0000 UTC m=+0.683341387 container remove 05c41517a1d60568a6b314134490bca3f76e4a52244835fa0a8076df2d5a686e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:24.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:24 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:24.371+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:24.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:24 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:24.371+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:24.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:24 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:24.121+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:24.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:24 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:24.121+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:24.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:24 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:24.121+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:24.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:24.318+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:24.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:24.318+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:24.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:24.246+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:24.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:24.246+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:24.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:24.246+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:24.617 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:24 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:24.343+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:24.617 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:24 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:24.343+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:24.617 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:24 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:24.343+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:24.617 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.247398301 +0000 UTC m=+0.017236892 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:24.618 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.398490506 +0000 UTC m=+0.168329137 container create c8092e65eb9e705978f88d7e6a5a6d8c05da3f45d2d3042f2ff05b0e2170652f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-17T04:57:24.618 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.617927097 +0000 UTC m=+0.387765689 container init c8092e65eb9e705978f88d7e6a5a6d8c05da3f45d2d3042f2ff05b0e2170652f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:57:24.618 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:24 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:24.334+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:24.618 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:24 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:24.334+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:24.915 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.625449633 +0000 UTC m=+0.395288224 container start c8092e65eb9e705978f88d7e6a5a6d8c05da3f45d2d3042f2ff05b0e2170652f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-17T04:57:24.915 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.73451818 +0000 UTC m=+0.504356792 container attach c8092e65eb9e705978f88d7e6a5a6d8c05da3f45d2d3042f2ff05b0e2170652f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-17T04:57:25.212 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 17 04:57:24 smithi110 podman[73936]: 2024-05-17 04:57:24.916436102 +0000 UTC m=+0.686274689 container died c8092e65eb9e705978f88d7e6a5a6d8c05da3f45d2d3042f2ff05b0e2170652f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-17T04:57:25.504 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:25 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:25.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:25.504 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:25 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:25.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:25.504 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:25 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:25.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:25.505 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:25 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:25.309+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:25.505 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:25 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:25.309+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:25.518 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-17T04:57:25.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:25.557 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-17T04:57:25.557 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-17T04:57:25.558 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.6 2024-05-17T04:57:25.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:25 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:25.332+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:25.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:25 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:25.332+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:25.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:25 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:25.131+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:25.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:25 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:25.131+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:25.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:25 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:25.131+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:25.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:25.205+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:25.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:25.205+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:25.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:25.205+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:25.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:25.289+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:25.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:25.289+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:25.891 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 systemd[1]: Stopping Ceph osd.6 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:26.239 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:25.891+0000 7f2c56fed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:26.239 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:25.891+0000 7f2c56fed700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-17T04:57:26.239 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:25 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:25.891+0000 7f2c56fed700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:26.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:26 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:26.324+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:26.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:26 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:26.324+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:26.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:26 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:26.144+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:26.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:26 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:26.144+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:26.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:26 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:26.144+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:26.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:26 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:26.284+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:26.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:26 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:26.284+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:26.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:26 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:26.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:26.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:26 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:26.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:26.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:26 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:26.346+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:26.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:26.312+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:26.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:26.312+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:26.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:26.239+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:26.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:26.239+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:26.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:26 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:26.239+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:27.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:27.186+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:27.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:27.186+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:27.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:27.186+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:27.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:27.371+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:27.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:27.371+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:27.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:27 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:27.371+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:27.257+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:27.257+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:27.257+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:27.385+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:27.385+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:27.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:27 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:27.385+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:27.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:27.283+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:27.576 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:27.283+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:27.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:27.286+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:27.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:27.286+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:27.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:27.286+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:28.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:28.173+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:28.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:28.173+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:28.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:28.173+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:28.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:28.366+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:28.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:28.366+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:28.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:28 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:28.366+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:28.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:28.300+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:28.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:28.300+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:28.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:28.300+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:28.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:28.334+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:28.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:28.334+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:28.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:28 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:28.334+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:28.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:28.331+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:28.575 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:28.331+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:28.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:28.322+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:28.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:28.322+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:28.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:28.322+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:28.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:28 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:28.322+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:29.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:29.339+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:29.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:29.339+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:29.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:29.339+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:29.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:29.154+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:29.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:29.154+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:29.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:29.154+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:29.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:29.154+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:29.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:29.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:29.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:29.358+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:29.358+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:29.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:29 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:29.358+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:29.825 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:29.371+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:29.371+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:29.371+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.962583+0000 front 2024-05-17T04:57:03.962654+0000 (oldest deadline 2024-05-17T04:57:29.262808+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:29.343+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:29.343+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:29.343+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:29.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:29 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:29.343+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:30.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:30.346+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:30.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:30.346+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:30.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:30.346+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:30.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:30.110+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:30.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:30.110+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:30.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:30.110+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:30.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:30 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:30.110+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:30.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:30.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:30.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:30.384+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:30.384+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:30.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:30 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:30.384+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:30.727 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:30.413+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.058959+0000 front 2024-05-17T04:56:47.059020+0000 (oldest deadline 2024-05-17T04:57:12.359037+0000) 2024-05-17T04:57:30.727 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:30.413+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.460434+0000 front 2024-05-17T04:56:53.460404+0000 (oldest deadline 2024-05-17T04:57:18.760629+0000) 2024-05-17T04:57:30.727 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6[57319]: 2024-05-17T04:57:30.413+0000 7f2c535a3700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.962583+0000 front 2024-05-17T04:57:03.962654+0000 (oldest deadline 2024-05-17T04:57:29.262808+0000) 2024-05-17T04:57:30.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:30.383+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:30.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:30.383+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:30.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:30.383+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:30.728 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:30 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:30.383+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:31.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:31.333+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:31.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:31.333+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:31.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:31.333+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:31.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:31.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:31.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:31.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:31.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:31.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:31.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:31.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:31.258+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:31.258+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:31.258+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:31.375+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:31.375+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:31.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:31 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:31.375+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:31.667 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:31 smithi050 podman[100192]: 2024-05-17 04:57:31.404130365 +0000 UTC m=+5.635251498 container died 9b18207624ddd0e9ed67191688798e43d29278a76e40215eca69383d96c9ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:57:31.667 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:31.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:31.667 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:31.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:31.667 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:31.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:31.667 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:31 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:31.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:32.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:31 smithi050 podman[100192]: 2024-05-17 04:57:31.774612299 +0000 UTC m=+6.005733431 container cleanup 9b18207624ddd0e9ed67191688798e43d29278a76e40215eca69383d96c9ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T04:57:32.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:31 smithi050 bash[100192]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6 2024-05-17T04:57:32.325 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100208]: 2024-05-17 04:57:32.084582154 +0000 UTC m=+0.674786278 container remove 9b18207624ddd0e9ed67191688798e43d29278a76e40215eca69383d96c9ff38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-17T04:57:32.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:32.352+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:32.352+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:32.352+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:32.118+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:32.118+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:32.118+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:32.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:32 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:32.118+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:32.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:32.253+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:32.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:32.253+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:32.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:32.253+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:32.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:32.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:32.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:32.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:32.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:32 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:32.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100304]: 2024-05-17 04:57:32.334635299 +0000 UTC m=+0.018127393 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100304]: 2024-05-17 04:57:32.46910485 +0000 UTC m=+0.152596944 container create 9792d75a0a7b399d6f48f1e1c4c41a24a5c4a0b1b0b1ad577a63a378544dcc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:32 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:32.349+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:32 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:32.349+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:32 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:32.349+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:32.615 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:32 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:32.349+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:33.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100304]: 2024-05-17 04:57:32.738437141 +0000 UTC m=+0.421929220 container init 9792d75a0a7b399d6f48f1e1c4c41a24a5c4a0b1b0b1ad577a63a378544dcc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-17T04:57:33.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100304]: 2024-05-17 04:57:32.742213791 +0000 UTC m=+0.425705879 container start 9792d75a0a7b399d6f48f1e1c4c41a24a5c4a0b1b0b1ad577a63a378544dcc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-17T04:57:33.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:32 smithi050 podman[100304]: 2024-05-17 04:57:32.821864416 +0000 UTC m=+0.505356514 container attach 9792d75a0a7b399d6f48f1e1c4c41a24a5c4a0b1b0b1ad577a63a378544dcc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:33.064 INFO:journalctl@ceph.osd.6.smithi050.stdout:May 17 04:57:33 smithi050 podman[100304]: 2024-05-17 04:57:33.065145298 +0000 UTC m=+0.748637395 container died 9792d75a0a7b399d6f48f1e1c4c41a24a5c4a0b1b0b1ad577a63a378544dcc55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-17T04:57:33.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:33.309+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:33.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:33.309+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:33.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:33.309+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:33.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:33.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:33.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:33.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:33.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:33.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:33.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:33 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:33.119+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:33.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:33.281+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:33.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:33.281+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:33.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:33.282+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:33.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:33.439+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:33.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:33.439+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:33.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:33 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:33.439+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:33.791 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:33.358+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:33.792 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:33.358+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:33.792 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:33.358+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:33.792 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:33 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:33.358+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:33.823 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-17T04:57:33.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:33.907 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-17T04:57:33.908 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-17T04:57:33.908 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.7 2024-05-17T04:57:34.231 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:33 smithi105 systemd[1]: Stopping Ceph osd.7 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:34.231 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:34.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:34.231 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:34.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:34.231 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:34.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:34.231 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:34.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:34.497 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.230+0000 7f96be47d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:34.497 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.230+0000 7f96be47d700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-17T04:57:34.498 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.230+0000 7f96be47d700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:34.498 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.357+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:34.498 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.357+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:34.498 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:34 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:34.357+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:34.294+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:34.294+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:34.294+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:34.449+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:34.449+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:34.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:34 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:34.449+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:34.575 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:34.320+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:34.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:34.320+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:34.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:34.320+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:34.576 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:34 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:34.320+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:35.386 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:35.070+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:35.387 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:35.070+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:35.387 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:35.070+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:35.387 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:35.070+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:35.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:35.256+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:35.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:35.256+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:35.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:35.256+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:35.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:35.469+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:35.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:35.469+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:35.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:35 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:35.469+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:35.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:35.386+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:35.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:35.386+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:35.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:35.386+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:35.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:35.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:35.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:35.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:35.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:35.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:35.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:35 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:35.348+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:36.419 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:36.111+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:36.419 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:36.111+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:36.419 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:36.111+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:36.419 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:36.111+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:36.419 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:36.111+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:36.298+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:36.298+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:36.298+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:36.494+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:36.494+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:36.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:36 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:36.494+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:36.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:36.418+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:36.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:36.418+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:36.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:36 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:36.418+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:36.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:36.377+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:36.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:36.377+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:36.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:36.377+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:36.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:36 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:36.377+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:37.379 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:37.125+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:37.379 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:37.125+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:37.380 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:37.125+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:37.380 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:37.125+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:37.380 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:37.125+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:37.265+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:37.265+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:37.265+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:37.452+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:37.452+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:37.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:37 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:37.452+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:37.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:37.379+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:37.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:37.379+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:37.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:37.379+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:37.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:37 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:37.379+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.555722+0000 front 2024-05-17T04:57:12.555750+0000 (oldest deadline 2024-05-17T04:57:36.655873+0000) 2024-05-17T04:57:37.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:37.391+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:37.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:37.391+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:37.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:37.391+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:37.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:37.391+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:37.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:37 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:37.391+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:38.382 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:38.090+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:38.382 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:38.090+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:38.382 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:38.090+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:38.382 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:38.090+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:38.382 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:38.090+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:38.271+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:38.271+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:38.271+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:38.271+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:38.477+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:38.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:38.477+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:38.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:38.477+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:38.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:38 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:38.477+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:38.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:38.382+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:47.451755+0000 front 2024-05-17T04:56:47.451853+0000 (oldest deadline 2024-05-17T04:57:13.351860+0000) 2024-05-17T04:57:38.814 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:38.382+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.853182+0000 front 2024-05-17T04:56:53.853324+0000 (oldest deadline 2024-05-17T04:57:23.253831+0000) 2024-05-17T04:57:38.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:38.382+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:03.254285+0000 front 2024-05-17T04:57:03.254235+0000 (oldest deadline 2024-05-17T04:57:26.654939+0000) 2024-05-17T04:57:38.815 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:38 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7[52762]: 2024-05-17T04:57:38.382+0000 7f96baa33700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.555722+0000 front 2024-05-17T04:57:12.555750+0000 (oldest deadline 2024-05-17T04:57:36.655873+0000) 2024-05-17T04:57:38.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:38.411+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:38.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:38.411+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:38.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:38.411+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:38.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:38.411+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:38.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:38 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:38.411+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:39.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:39.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:39.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:39.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:39.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:39.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:39.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:39.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:39.565 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:39.087+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:39.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:39.272+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:39.566 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:39.272+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:39.272+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:39.272+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:39.446+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:39.446+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:39.446+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:39.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:39 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:39.446+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:39.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:39 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:39.379+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:39.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:39 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:39.379+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:39.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:39 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:39.379+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:39.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:39 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:39.379+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:39.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:39 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:39.379+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:39.982 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:39 smithi105 podman[73289]: 2024-05-17 04:57:39.713411606 +0000 UTC m=+5.606381281 container died 62bc3bdce0fcf5c09e54ea642941cdeaa1c11f59535116c0b0442554da742ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0) 2024-05-17T04:57:40.314 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73289]: 2024-05-17 04:57:40.064857617 +0000 UTC m=+5.957827303 container cleanup 62bc3bdce0fcf5c09e54ea642941cdeaa1c11f59535116c0b0442554da742ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-17T04:57:40.314 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 bash[73289]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7 2024-05-17T04:57:40.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:40 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:40.055+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:40.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:40 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:40.055+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:40.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:40 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:40.055+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:40.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:40 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:40.055+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:40.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:40 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:40.055+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:40.433+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:40.433+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:40.433+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:40.433+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:40.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:40.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:40.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:40.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:40 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:40.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:40.691 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73319]: 2024-05-17 04:57:40.37156853 +0000 UTC m=+0.655449300 container remove 62bc3bdce0fcf5c09e54ea642941cdeaa1c11f59535116c0b0442554da742ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:57:40.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:40 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:40.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:40.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:40 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:40.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:40.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:40 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:40.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:40.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:40 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:40.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:40.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:40 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:40.336+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:40.985 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73408]: 2024-05-17 04:57:40.692352576 +0000 UTC m=+0.112431886 container create 78336d77f9139e93f88a4b05cc54956b8dbbf9ffeb76dfb0a45ff2809bd37873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-17T04:57:40.986 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73408]: 2024-05-17 04:57:40.597135515 +0000 UTC m=+0.017214786 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:41.284 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73408]: 2024-05-17 04:57:40.986725582 +0000 UTC m=+0.406804849 container init 78336d77f9139e93f88a4b05cc54956b8dbbf9ffeb76dfb0a45ff2809bd37873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:40 smithi105 podman[73408]: 2024-05-17 04:57:40.990163692 +0000 UTC m=+0.410242963 container start 78336d77f9139e93f88a4b05cc54956b8dbbf9ffeb76dfb0a45ff2809bd37873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:41 smithi105 podman[73408]: 2024-05-17 04:57:41.120185958 +0000 UTC m=+0.540265235 container attach 78336d77f9139e93f88a4b05cc54956b8dbbf9ffeb76dfb0a45ff2809bd37873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:41 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:41.051+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:41 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:41.051+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:41 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:41.051+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:41 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:41.051+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:41.285 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:41 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:41.051+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:41.564 INFO:journalctl@ceph.osd.7.smithi105.stdout:May 17 04:57:41 smithi105 podman[73408]: 2024-05-17 04:57:41.285430938 +0000 UTC m=+0.705510211 container died 78336d77f9139e93f88a4b05cc54956b8dbbf9ffeb76dfb0a45ff2809bd37873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:41.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:41.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:41.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:41.302+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:41.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:41.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:41.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:41.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:41 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:41.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:41.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:41 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:41.386+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:41.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:41 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:41.386+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:41.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:41 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:41.386+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:41.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:41 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:41.386+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:41.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:41 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:41.386+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:41.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-17T04:57:42.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:42.015 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-17T04:57:42.015 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-17T04:57:42.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.8 2024-05-17T04:57:42.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:42 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:42.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:42.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:42 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:42.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:42.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:42 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:42.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:42.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:42 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:42.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:42.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:42 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:42.032+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:42.316 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 systemd[1]: Stopping Ceph osd.8 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:42.317 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:42.317 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:42.317 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:42.317 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.288+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:42.816 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.349+0000 7fbcf8c34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.349+0000 7fbcf8c34700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:42.349+0000 7fbcf8c34700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:42.502+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:42.502+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:42.502+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:42.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:42 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:42.502+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:42.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:42.382+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:42.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:42.382+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:42.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:42.382+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:42.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:42.382+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:42.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:42 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:42.382+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:43.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:43 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:43.049+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:43.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:43 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:43.049+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:43.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:43 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:43.049+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:43.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:43 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:43.049+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:43.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:43 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:43.049+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:43.314+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:43.314+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:43.314+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:43.314+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:43.492+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:43.492+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:43.492+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:43.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:43 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:43.492+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:43.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:43.385+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:43.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:43.385+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:43.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:43.385+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:43.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:43.385+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:43.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:43 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:43.385+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:44.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:44.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:44.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:44.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:44.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.024+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:44.326+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:44.326+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:44.326+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:44.326+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:44.516+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:44.516+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:44.516+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:44.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:44.516+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:44.818 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:44.516+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:44.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:44.434+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:44.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:44.434+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:44.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:44.434+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:44.827 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:44.434+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:44.827 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:44 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:44.434+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:45.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.975+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:45.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.975+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:45.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.975+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:45.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.975+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:45.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:44 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:44.975+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:45.522 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:45.325+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:45.522 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:45.325+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:45.522 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:45.325+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:45.522 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:45.325+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:45.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:45.522+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:45.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:45.522+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:45.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:45.522+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:45.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:45.522+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:45.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:45 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:45.522+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:45.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:45.392+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:45.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:45.392+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:45.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:45.392+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:45.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:45.392+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:45.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:45 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:45.392+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:46.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:46.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:46.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:46.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:46.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:46.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:45 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:45.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:46.482+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:46.482+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:46.482+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:46.482+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:46.482+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:46.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:46.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:46.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:46.568 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:46.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:46.568 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:46.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:46.568 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:46 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:46.308+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:20.021483+0000 front 2024-05-17T04:57:20.021527+0000 (oldest deadline 2024-05-17T04:57:45.921624+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:46.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:46 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:46.428+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:47.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:47.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:47.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:47.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:47.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:47.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:46 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:46.917+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:47.278+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.115297+0000 front 2024-05-17T04:56:48.115525+0000 (oldest deadline 2024-05-17T04:57:10.415383+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:47.278+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:56.817442+0000 front 2024-05-17T04:56:56.817417+0000 (oldest deadline 2024-05-17T04:57:21.517696+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:47.278+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:02.019058+0000 front 2024-05-17T04:57:02.018749+0000 (oldest deadline 2024-05-17T04:57:26.718864+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:47.278+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:14.320300+0000 front 2024-05-17T04:57:14.320468+0000 (oldest deadline 2024-05-17T04:57:37.820221+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8[54483]: 2024-05-17T04:57:47.278+0000 7fbcf11e2700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:20.021483+0000 front 2024-05-17T04:57:20.021527+0000 (oldest deadline 2024-05-17T04:57:45.921624+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:47.485+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:47.485+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:47.485+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:47.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:47.485+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:47.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:47.485+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:47.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:47 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:47.378+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:48.141 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:47 smithi110 podman[74055]: 2024-05-17 04:57:47.853779629 +0000 UTC m=+5.639355726 container died 9a3ecf4a23a178195fada8a357dd2841f0df48f46e8d0f4cf6a64e148e83696f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:57:48.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:48.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:48.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:48.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:48.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:48.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:47 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:47.939+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:48.489 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:48 smithi110 podman[74055]: 2024-05-17 04:57:48.141890075 +0000 UTC m=+5.927466220 container cleanup 9a3ecf4a23a178195fada8a357dd2841f0df48f46e8d0f4cf6a64e148e83696f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-17T04:57:48.489 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:48 smithi110 bash[74055]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8 2024-05-17T04:57:48.489 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:48 smithi110 podman[74071]: 2024-05-17 04:57:48.326407015 +0000 UTC m=+0.470585577 container remove 9a3ecf4a23a178195fada8a357dd2841f0df48f46e8d0f4cf6a64e148e83696f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-05-17T04:57:48.752 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:48 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:48.489+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:48.753 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:48 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:48.489+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:48.753 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:48 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:48.489+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:48.753 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:48 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:48.489+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:48.753 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:48 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:48.489+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:48.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:48.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:48.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:48.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:48.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:48.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:48 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:48.423+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:49.004 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:48 smithi110 podman[74158]: 2024-05-17 04:57:48.656204252 +0000 UTC m=+0.017340554 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:49.004 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:48 smithi110 podman[74158]: 2024-05-17 04:57:48.760640012 +0000 UTC m=+0.121776336 container create 6416a055b88d6afa12de84413b1fa145aafe4c167239c8e6b0cda9eec4e53d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-17T04:57:49.004 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:49 smithi110 podman[74158]: 2024-05-17 04:57:49.005351958 +0000 UTC m=+0.366488240 container init 6416a055b88d6afa12de84413b1fa145aafe4c167239c8e6b0cda9eec4e53d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T04:57:49.302 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:49 smithi110 podman[74158]: 2024-05-17 04:57:49.008752036 +0000 UTC m=+0.369888326 container start 6416a055b88d6afa12de84413b1fa145aafe4c167239c8e6b0cda9eec4e53d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:57:49.302 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:49 smithi110 podman[74158]: 2024-05-17 04:57:49.088617583 +0000 UTC m=+0.449753877 container attach 6416a055b88d6afa12de84413b1fa145aafe4c167239c8e6b0cda9eec4e53d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0) 2024-05-17T04:57:49.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.962+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:49.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.962+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:49.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.962+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:49.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.963+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:49.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.963+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:49.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:48 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:48.963+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 17 04:57:49 smithi110 podman[74158]: 2024-05-17 04:57:49.303056549 +0000 UTC m=+0.664192836 container died 6416a055b88d6afa12de84413b1fa145aafe4c167239c8e6b0cda9eec4e53d7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:49 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:49.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:49 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:49.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:49 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:49.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:49 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:49.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:49.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:49 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:49.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:49.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:49.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:49.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:49.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:49.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:49.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:49.464+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:49.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-17T04:57:49.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:49.911 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-17T04:57:49.911 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-17T04:57:49.911 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.9 2024-05-17T04:57:50.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:50.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:50.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:50.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:50.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:50.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:49 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:49.930+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:50.325 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:49 smithi050 systemd[1]: Stopping Ceph osd.9 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:50.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:50 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:50.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:50.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:50 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:50.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:50.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:50 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:50.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:50.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:50 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:50.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:50.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:50 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:50.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:50.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.361+0000 7f08b712e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.361+0000 7f08b712e700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.361+0000 7f08b712e700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:50.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:50 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:50.473+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:51.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:51.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:51.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:51.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:51.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:51.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:50 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:50.937+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:51.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:51.458+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:51.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:51.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:51.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:51.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:51.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:51.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:51 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:51.471+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:52.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:52.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:52.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:52.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:52.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:52.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:51 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:51.932+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:52.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:52 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:52.471+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:52.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:52.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:52.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:52.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:52.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:52.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:52 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:52.491+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:53.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:53.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:53.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:53.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:53.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:53.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:53.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:52 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:52.928+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:53.816 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:53.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:53.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:53.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:53.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:53.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:53 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:53.483+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:53.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:53.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:53.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:53.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:53.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:53.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:53 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:53.463+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:54.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:54.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:54.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:54.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:54.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:54.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:54.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:53 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:53.945+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:54.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:54 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:54.443+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:54.825 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:39.953369+0000 front 2024-05-17T04:56:39.953306+0000 (oldest deadline 2024-05-17T04:57:01.653534+0000) 2024-05-17T04:57:54.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.655675+0000 front 2024-05-17T04:56:48.655902+0000 (oldest deadline 2024-05-17T04:57:13.355643+0000) 2024-05-17T04:57:54.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.356274+0000 front 2024-05-17T04:56:53.356247+0000 (oldest deadline 2024-05-17T04:57:19.256233+0000) 2024-05-17T04:57:54.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.357475+0000 front 2024-05-17T04:57:00.357500+0000 (oldest deadline 2024-05-17T04:57:27.358681+0000) 2024-05-17T04:57:54.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:13.160259+0000 front 2024-05-17T04:57:13.160501+0000 (oldest deadline 2024-05-17T04:57:37.260144+0000) 2024-05-17T04:57:54.826 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:54 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9[64012]: 2024-05-17T04:57:54.494+0000 7f08af6dc700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:17.260801+0000 front 2024-05-17T04:57:17.260994+0000 (oldest deadline 2024-05-17T04:57:46.061257+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:55.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:54 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:54.948+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:55.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:55.409+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:56.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:56.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:55 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:55.909+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:56.325 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:55 smithi050 podman[100412]: 2024-05-17 04:57:55.978912322 +0000 UTC m=+5.855376303 container died 3e1bea69b00a5729347ccce85b58878f05d9d0d4537216b9b199bcd245435aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-17T04:57:56.704 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:56 smithi050 podman[100412]: 2024-05-17 04:57:56.461968637 +0000 UTC m=+6.338432618 container cleanup 3e1bea69b00a5729347ccce85b58878f05d9d0d4537216b9b199bcd245435aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-17T04:57:56.704 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:56 smithi050 bash[100412]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:56.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:56 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:56.393+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:57.043 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:56 smithi050 podman[100429]: 2024-05-17 04:57:56.7049263 +0000 UTC m=+0.723063317 container remove 3e1bea69b00a5729347ccce85b58878f05d9d0d4537216b9b199bcd245435aea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-17T04:57:57.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:57.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:56 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:56.926+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:57.326 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:56.947151633 +0000 UTC m=+0.018225714 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:57:57.326 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:57.055899052 +0000 UTC m=+0.126973110 container create 2c7fce8f25ad884a7d4ac246ccbe3c7b07f7e3a2cacbe7b177ffd989bf5fa7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-17T04:57:57.326 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:57.266960268 +0000 UTC m=+0.338034325 container init 2c7fce8f25ad884a7d4ac246ccbe3c7b07f7e3a2cacbe7b177ffd989bf5fa7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:57:57.326 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:57.27044782 +0000 UTC m=+0.341521890 container start 2c7fce8f25ad884a7d4ac246ccbe3c7b07f7e3a2cacbe7b177ffd989bf5fa7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-17T04:57:57.580 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:57.367047718 +0000 UTC m=+0.438121821 container attach 2c7fce8f25ad884a7d4ac246ccbe3c7b07f7e3a2cacbe7b177ffd989bf5fa7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:57.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:57 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:57.407+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:57.861 INFO:journalctl@ceph.osd.9.smithi050.stdout:May 17 04:57:57 smithi050 podman[100518]: 2024-05-17 04:57:57.581659222 +0000 UTC m=+0.652733300 container died 2c7fce8f25ad884a7d4ac246ccbe3c7b07f7e3a2cacbe7b177ffd989bf5fa7a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-17T04:57:58.222 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-17T04:57:58.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:57:58.266 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-17T04:57:58.266 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-17T04:57:58.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.10 2024-05-17T04:57:58.293 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:58.293 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:58.293 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:58.294 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:58.294 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:58.294 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:58.294 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:57 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:57.969+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:58.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 systemd[1]: Stopping Ceph osd.10 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:58.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:58 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:58.374+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:57:58.993 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.575+0000 7f9514594700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:57:58.993 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.575+0000 7f9514594700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-17T04:57:58.993 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.575+0000 7f9514594700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:57:59.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:57:59.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:57:59.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:57:59.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:57:59.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:57:59.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:57:59.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:57:58 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:57:58.994+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:57:59.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:57:59 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:57:59.411+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:00.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:58:00.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:58:00.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:58:00.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:58:00.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:58:00.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:58:00.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:00 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:00.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:00.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:00 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:00.438+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:01.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:58:01.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:58:01.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:58:01.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:58:01.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:58:01.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:58:01.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:01 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:01.050+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:01.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:01 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:01.472+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:02.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:58:02.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:58:02.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:58:02.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:58:02.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:58:02.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:58:02.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:02 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:02.018+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:58:02.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:02.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:02.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:02.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:02.567 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:02.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:02.568 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:02 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:02.463+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:03.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:36.889913+0000 front 2024-05-17T04:56:36.890009+0000 (oldest deadline 2024-05-17T04:57:02.190074+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:48.592018+0000 front 2024-05-17T04:56:48.591982+0000 (oldest deadline 2024-05-17T04:57:10.292219+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:57.294088+0000 front 2024-05-17T04:56:57.294180+0000 (oldest deadline 2024-05-17T04:57:20.794161+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6814 osd.3 since back 2024-05-17T04:57:00.794779+0000 front 2024-05-17T04:57:00.794722+0000 (oldest deadline 2024-05-17T04:57:28.395331+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:11.296491+0000 front 2024-05-17T04:57:11.296445+0000 (oldest deadline 2024-05-17T04:57:35.396495+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:22.397797+0000 front 2024-05-17T04:57:22.397916+0000 (oldest deadline 2024-05-17T04:57:45.897657+0000) 2024-05-17T04:58:03.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:03 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10[59768]: 2024-05-17T04:58:03.039+0000 7f9510b4a700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:27.598822+0000 front 2024-05-17T04:57:27.598785+0000 (oldest deadline 2024-05-17T04:57:52.298654+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:03.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:03 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:03.455+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:04.301 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:04 smithi105 podman[73512]: 2024-05-17 04:58:04.026735227 +0000 UTC m=+5.567284497 container died adee554ee033a4b5dbc644293c8c4c79ba9c482b4fdc3281b2cf0f258607c12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:04.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:04 smithi105 podman[73512]: 2024-05-17 04:58:04.409646745 +0000 UTC m=+5.950195992 container cleanup adee554ee033a4b5dbc644293c8c4c79ba9c482b4fdc3281b2cf0f258607c12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-17T04:58:04.564 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:04 smithi105 bash[73512]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10 2024-05-17T04:58:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:04.718 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:04.719 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:04.719 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:04 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:04.466+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:05.024 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:04 smithi105 podman[73530]: 2024-05-17 04:58:04.66946191 +0000 UTC m=+0.638976123 container remove adee554ee033a4b5dbc644293c8c4c79ba9c482b4fdc3281b2cf0f258607c12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:05.314 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:04.928032817 +0000 UTC m=+0.017941569 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:58:05.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:05.036862953 +0000 UTC m=+0.126771686 container create 79bcafc7258b1391692134ea3e7a5ed1c347ea6a8b6bd106b7e3e443cd9c9cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514) 2024-05-17T04:58:05.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:05.264105639 +0000 UTC m=+0.354014390 container init 79bcafc7258b1391692134ea3e7a5ed1c347ea6a8b6bd106b7e3e443cd9c9cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:05.315 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:05.267459689 +0000 UTC m=+0.357368429 container start 79bcafc7258b1391692134ea3e7a5ed1c347ea6a8b6bd106b7e3e443cd9c9cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:58:05.814 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:05.358646758 +0000 UTC m=+0.448555509 container attach 79bcafc7258b1391692134ea3e7a5ed1c347ea6a8b6bd106b7e3e443cd9c9cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514) 2024-05-17T04:58:05.814 INFO:journalctl@ceph.osd.10.smithi105.stdout:May 17 04:58:05 smithi105 podman[73617]: 2024-05-17 04:58:05.560502056 +0000 UTC m=+0.650410795 container died 79bcafc7258b1391692134ea3e7a5ed1c347ea6a8b6bd106b7e3e443cd9c9cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:05.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:05 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:05.459+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:06.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-17T04:58:06.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:06.246 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-17T04:58:06.246 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-17T04:58:06.247 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.11 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 systemd[1]: Stopping Ceph osd.11 for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:06.553 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:06.554 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.504+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:06.554 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.552+0000 7f44db851700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:06.554 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.552+0000 7f44db851700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-17T04:58:06.554 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:06 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:06.552+0000 7f44db851700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:07.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:07 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:07.545+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:08.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:08 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:08.549+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-05-17T04:57:43.506325+0000 front 2024-05-17T04:57:43.506348+0000 (oldest deadline 2024-05-17T04:58:08.206394+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:09.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:09 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:09.503+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-05-17T04:57:43.506325+0000 front 2024-05-17T04:57:43.506348+0000 (oldest deadline 2024-05-17T04:58:08.206394+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:10.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:10 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:10.505+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-05-17T04:57:43.506325+0000 front 2024-05-17T04:57:43.506348+0000 (oldest deadline 2024-05-17T04:58:08.206394+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6806 osd.0 since back 2024-05-17T04:56:38.295810+0000 front 2024-05-17T04:56:38.295834+0000 (oldest deadline 2024-05-17T04:57:01.795800+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-05-17T04:56:46.397748+0000 front 2024-05-17T04:56:46.397713+0000 (oldest deadline 2024-05-17T04:57:13.398414+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-17T04:56:53.398985+0000 front 2024-05-17T04:56:53.398840+0000 (oldest deadline 2024-05-17T04:57:18.699003+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-05-17T04:57:12.201220+0000 front 2024-05-17T04:57:12.201163+0000 (oldest deadline 2024-05-17T04:57:38.101108+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-17T04:57:18.601847+0000 front 2024-05-17T04:57:18.601919+0000 (oldest deadline 2024-05-17T04:57:44.501888+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.50:6822 osd.6 since back 2024-05-17T04:57:26.703658+0000 front 2024-05-17T04:57:26.703673+0000 (oldest deadline 2024-05-17T04:57:51.403428+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-05-17T04:57:36.104450+0000 front 2024-05-17T04:57:36.104506+0000 (oldest deadline 2024-05-17T04:58:01.304892+0000) 2024-05-17T04:58:11.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:11 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11[61492]: 2024-05-17T04:58:11.527+0000 7f44d3dff700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-05-17T04:57:43.506325+0000 front 2024-05-17T04:57:43.506348+0000 (oldest deadline 2024-05-17T04:58:08.206394+0000) 2024-05-17T04:58:12.317 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:12 smithi110 podman[74275]: 2024-05-17 04:58:12.057304418 +0000 UTC m=+5.610202076 container died e66c77b80e78dbdb9b21c1893057f7394b45383cddb6389faa3594e14ded605c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:12.816 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:12 smithi110 podman[74275]: 2024-05-17 04:58:12.403448732 +0000 UTC m=+5.956346389 container cleanup e66c77b80e78dbdb9b21c1893057f7394b45383cddb6389faa3594e14ded605c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-17T04:58:12.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:12 smithi110 bash[74275]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11 2024-05-17T04:58:12.817 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:12 smithi110 podman[74292]: 2024-05-17 04:58:12.579903717 +0000 UTC m=+0.524788059 container remove e66c77b80e78dbdb9b21c1893057f7394b45383cddb6389faa3594e14ded605c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-17T04:58:13.269 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:12.92055493 +0000 UTC m=+0.016809167 image pull 8880f7613d0283efa7002505fdccdbc3e37d3b5da31c57fa22d62262c5dab66c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b 2024-05-17T04:58:13.270 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:13.064287611 +0000 UTC m=+0.160541871 container create a8127c327e2c3097e08fb0d7d9937d19a338f02461f1741e2de1a3a8abee9435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:13.270 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:13.267107396 +0000 UTC m=+0.363361646 container init a8127c327e2c3097e08fb0d7d9937d19a338f02461f1741e2de1a3a8abee9435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-17T04:58:13.270 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:13.270448558 +0000 UTC m=+0.366702805 container start a8127c327e2c3097e08fb0d7d9937d19a338f02461f1741e2de1a3a8abee9435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T04:58:13.566 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:13.366723265 +0000 UTC m=+0.462977504 container attach a8127c327e2c3097e08fb0d7d9937d19a338f02461f1741e2de1a3a8abee9435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:13.860 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 17 04:58:13 smithi110 podman[74378]: 2024-05-17 04:58:13.568161416 +0000 UTC m=+0.664415659 container died a8127c327e2c3097e08fb0d7d9937d19a338f02461f1741e2de1a3a8abee9435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-17T04:58:14.155 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-17T04:58:14.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:14.196 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-17T04:58:14.196 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-17T04:58:14.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.a 2024-05-17T04:58:14.532 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:14 smithi050 systemd[1]: Stopping Ceph mds.a for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:14.825 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-a[71015]: 2024-05-17T04:58:14.531+0000 7f6ec1feb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:14.826 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:14 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-a[71015]: 2024-05-17T04:58:14.531+0000 7f6ec1feb700 -1 mds.a *** got signal Terminated *** 2024-05-17T04:58:18.231 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:17 smithi050 podman[100648]: 2024-05-17 04:58:17.967796794 +0000 UTC m=+3.564455490 container died 05accb6677eca0b63d869a3f99c57f59125fbcc028e71cd5a739a2fbf54c1ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:18.576 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:18 smithi050 podman[100648]: 2024-05-17 04:58:18.323230542 +0000 UTC m=+3.919889225 container cleanup 05accb6677eca0b63d869a3f99c57f59125fbcc028e71cd5a739a2fbf54c1ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-17T04:58:18.576 INFO:journalctl@ceph.mds.a.smithi050.stdout:May 17 04:58:18 smithi050 bash[100648]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-a 2024-05-17T04:58:18.766 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-17T04:58:18.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:18.802 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-17T04:58:18.802 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-17T04:58:18.803 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.d 2024-05-17T04:58:19.136 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:18 smithi050 systemd[1]: Stopping Ceph mds.d for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:19.575 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-d[69743]: 2024-05-17T04:58:19.136+0000 7fe7a0b84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:19.576 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:19 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-d[69743]: 2024-05-17T04:58:19.136+0000 7fe7a0b84700 -1 mds.d *** got signal Terminated *** 2024-05-17T04:58:23.427 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:23 smithi050 podman[100770]: 2024-05-17 04:58:23.145426734 +0000 UTC m=+4.115337332 container died d5133a926436953260e5c4dcc71b8e8304c40e114e37281f1ba4997011ef2e3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-d, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-17T04:58:23.745 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:23 smithi050 podman[100770]: 2024-05-17 04:58:23.527720128 +0000 UTC m=+4.497630717 container cleanup d5133a926436953260e5c4dcc71b8e8304c40e114e37281f1ba4997011ef2e3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-17T04:58:23.745 INFO:journalctl@ceph.mds.d.smithi050.stdout:May 17 04:58:23 smithi050 bash[100770]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-d 2024-05-17T04:58:23.901 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-17T04:58:23.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:23.936 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-17T04:58:23.936 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-17T04:58:23.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.g 2024-05-17T04:58:24.257 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:24 smithi050 systemd[1]: Stopping Ceph mds.g for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:24.575 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g[70565]: 2024-05-17T04:58:24.257+0000 7f681a7ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:24.576 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:24 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g[70565]: 2024-05-17T04:58:24.257+0000 7f681a7ea700 -1 mds.g *** got signal Terminated *** 2024-05-17T04:58:26.174 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:25 smithi050 podman[100891]: 2024-05-17 04:58:25.877836861 +0000 UTC m=+1.706474836 container died bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:58:26.549 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:26 smithi050 podman[100891]: 2024-05-17 04:58:26.282413845 +0000 UTC m=+2.111051820 container cleanup bd18c3f0e4ae08e612dbf45d9bf794ccf61d108fdd0c940698cb32ca7728433c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:26.549 INFO:journalctl@ceph.mds.g.smithi050.stdout:May 17 04:58:26 smithi050 bash[100891]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-g 2024-05-17T04:58:26.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-17T04:58:26.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:26.760 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-17T04:58:26.760 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-17T04:58:26.760 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.j 2024-05-17T04:58:27.075 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:26 smithi050 systemd[1]: Stopping Ceph mds.j for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:27.576 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j[69303]: 2024-05-17T04:58:27.086+0000 7f573634f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:27.576 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:27 smithi050 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j[69303]: 2024-05-17T04:58:27.086+0000 7f573634f700 -1 mds.j *** got signal Terminated *** 2024-05-17T04:58:28.326 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:28 smithi050 podman[101010]: 2024-05-17 04:58:28.07202945 +0000 UTC m=+1.072387821 container died 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-17T04:58:28.826 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:28 smithi050 podman[101010]: 2024-05-17 04:58:28.437049043 +0000 UTC m=+1.437407418 container cleanup 2b852a49f7586a313685066fbd0f730523f72515c647cc410e8284bc92332812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:28.826 INFO:journalctl@ceph.mds.j.smithi050.stdout:May 17 04:58:28 smithi050 bash[101010]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-j 2024-05-17T04:58:28.968 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-17T04:58:29.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:29.001 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-17T04:58:29.002 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-17T04:58:29.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.b 2024-05-17T04:58:29.314 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:29 smithi105 systemd[1]: Stopping Ceph mds.b for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:29.814 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: 2024-05-17T04:58:29.385+0000 7fbc22d7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:29.814 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:29 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b[64942]: 2024-05-17T04:58:29.385+0000 7fbc22d7c700 -1 mds.b *** got signal Terminated *** 2024-05-17T04:58:30.814 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:30 smithi105 podman[73736]: 2024-05-17 04:58:30.500845137 +0000 UTC m=+1.293698391 container died 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:31.261 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:31 smithi105 podman[73736]: 2024-05-17 04:58:31.026826395 +0000 UTC m=+1.819679648 container cleanup 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-17T04:58:31.261 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:31 smithi105 bash[73736]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b 2024-05-17T04:58:31.513 INFO:journalctl@ceph.mds.b.smithi105.stdout:May 17 04:58:31 smithi105 podman[73750]: 2024-05-17 04:58:31.261800262 +0000 UTC m=+0.757354302 container remove 68bf080cc3aa2d615605bb1c5bdd9e71a6dde1c2fc095afdf34c97e50b941a64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-17T04:58:31.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-17T04:58:31.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:31.552 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-17T04:58:31.552 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-17T04:58:31.552 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.e 2024-05-17T04:58:31.814 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:31 smithi105 systemd[1]: Stopping Ceph mds.e for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:32.314 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: 2024-05-17T04:58:31.906+0000 7efde4220700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:32.314 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:31 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e[64330]: 2024-05-17T04:58:31.906+0000 7efde4220700 -1 mds.e *** got signal Terminated *** 2024-05-17T04:58:34.814 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:34 smithi105 podman[73857]: 2024-05-17 04:58:34.507345742 +0000 UTC m=+2.729967953 container died f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:35.287 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:34 smithi105 podman[73857]: 2024-05-17 04:58:34.831420584 +0000 UTC m=+3.054042797 container cleanup f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a) 2024-05-17T04:58:35.288 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:34 smithi105 bash[73857]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e 2024-05-17T04:58:35.288 INFO:journalctl@ceph.mds.e.smithi105.stdout:May 17 04:58:35 smithi105 podman[73869]: 2024-05-17 04:58:35.057700155 +0000 UTC m=+0.544270170 container remove f84decd82357bfcc6ddecb17ca60594df4dea3bfca3448532747e5488545262a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:35.302 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-17T04:58:35.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:35.335 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-17T04:58:35.335 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-17T04:58:35.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.h 2024-05-17T04:58:35.564 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:35 smithi105 systemd[1]: Stopping Ceph mds.h for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:36.064 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: 2024-05-17T04:58:35.651+0000 7fdcb6531700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:36.064 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:35 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h[64635]: 2024-05-17T04:58:35.651+0000 7fdcb6531700 -1 mds.h *** got signal Terminated *** 2024-05-17T04:58:38.814 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:38 smithi105 podman[73979]: 2024-05-17 04:58:38.497798958 +0000 UTC m=+2.943710254 container stop fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-17T04:58:38.815 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:38 smithi105 podman[73979]: 2024-05-17 04:58:38.523501956 +0000 UTC m=+2.969413260 container died fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True) 2024-05-17T04:58:39.102 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:38 smithi105 podman[73979]: 2024-05-17 04:58:38.935186571 +0000 UTC m=+3.381097875 container cleanup fdb48be1149d89f73992b57eb1b91f8c02fa648db0f818250b87329ba8f22520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-17T04:58:39.102 INFO:journalctl@ceph.mds.h.smithi105.stdout:May 17 04:58:38 smithi105 bash[73979]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-h 2024-05-17T04:58:39.319 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-17T04:58:39.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:39.355 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-17T04:58:39.355 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-17T04:58:39.355 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.k 2024-05-17T04:58:39.680 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:39 smithi105 systemd[1]: Stopping Ceph mds.k for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:40.064 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k[64015]: 2024-05-17T04:58:39.680+0000 7f4bb4f03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:40.064 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:39 smithi105 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k[64015]: 2024-05-17T04:58:39.680+0000 7f4bb4f03700 -1 mds.k *** got signal Terminated *** 2024-05-17T04:58:43.620 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:43 smithi105 podman[74111]: 2024-05-17 04:58:43.359670937 +0000 UTC m=+3.786303523 container died fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:43.931 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:43 smithi105 podman[74111]: 2024-05-17 04:58:43.714312526 +0000 UTC m=+4.140945101 container cleanup fed180584642bc5e75dd2f9c3a2b63c458822ad7f0331d132c098eec7f4ee6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-17T04:58:43.931 INFO:journalctl@ceph.mds.k.smithi105.stdout:May 17 04:58:43 smithi105 bash[74111]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-k 2024-05-17T04:58:44.075 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-17T04:58:44.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:44.108 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-17T04:58:44.108 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-17T04:58:44.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.c 2024-05-17T04:58:44.554 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:44 smithi110 systemd[1]: Stopping Ceph mds.c for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:44.816 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: 2024-05-17T04:58:44.553+0000 7fb1f1ff7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:44.817 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:44 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c[66348]: 2024-05-17T04:58:44.553+0000 7fb1f1ff7700 -1 mds.c *** got signal Terminated *** 2024-05-17T04:58:46.817 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:46 smithi110 podman[74483]: 2024-05-17 04:58:46.524897919 +0000 UTC m=+2.223022387 container died 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:47.312 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 podman[74483]: 2024-05-17 04:58:47.045558493 +0000 UTC m=+2.743682954 container cleanup 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-17T04:58:47.312 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 bash[74483]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c 2024-05-17T04:58:47.534 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-17T04:58:47.566 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 podman[74507]: 2024-05-17 04:58:47.312841108 +0000 UTC m=+0.783903730 container remove 92bad0877e04172474ad0193188f32d77032b3c2b71f06555f2ce8d23da62af0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:47.567 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-17T04:58:47.567 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 systemd[1]: Stopped Ceph mds.c for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:58:47.567 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 17 04:58:47 smithi110 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.c.service: Consumed 9min 37.383s CPU time. 2024-05-17T04:58:47.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:47.571 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-17T04:58:47.571 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-17T04:58:47.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.f 2024-05-17T04:58:47.958 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:47 smithi110 systemd[1]: Stopping Ceph mds.f for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:48.316 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: 2024-05-17T04:58:47.957+0000 7fc4f9712700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:48.316 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:47 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f[66048]: 2024-05-17T04:58:47.957+0000 7fc4f9712700 -1 mds.f *** got signal Terminated *** 2024-05-17T04:58:50.817 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:50 smithi110 podman[74615]: 2024-05-17 04:58:50.52629484 +0000 UTC m=+2.730286931 container died f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:58:51.266 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:50 smithi110 podman[74615]: 2024-05-17 04:58:50.850059205 +0000 UTC m=+3.054051295 container cleanup f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-17T04:58:51.266 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:50 smithi110 bash[74615]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f 2024-05-17T04:58:51.266 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 17 04:58:51 smithi110 podman[74628]: 2024-05-17 04:58:51.051270058 +0000 UTC m=+0.518603800 container remove f8b014e3ee7427b79832e6a71188dc65af650173e6fe47222bbfcb04d2ab405f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, ceph=True) 2024-05-17T04:58:51.278 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-17T04:58:51.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:51.311 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-17T04:58:51.311 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-17T04:58:51.311 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.i 2024-05-17T04:58:51.566 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:51 smithi110 systemd[1]: Stopping Ceph mds.i for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:52.066 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: 2024-05-17T04:58:51.653+0000 7f77665fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:52.066 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:51 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i[66650]: 2024-05-17T04:58:51.653+0000 7f77665fe700 -1 mds.i *** got signal Terminated *** 2024-05-17T04:58:54.816 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:54 smithi110 podman[74735]: 2024-05-17 04:58:54.522099264 +0000 UTC m=+2.986976789 container died 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True) 2024-05-17T04:58:55.316 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 podman[74735]: 2024-05-17 04:58:55.153914339 +0000 UTC m=+3.618791859 container cleanup 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-17T04:58:55.317 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 bash[74735]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i 2024-05-17T04:58:55.592 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-17T04:58:55.613 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 podman[74748]: 2024-05-17 04:58:55.346984567 +0000 UTC m=+0.817811071 container remove 8a4dee2609fc7b0c4d9a30f68577448532ed1cd8fb4c58f269ba1236d89ccd17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-17T04:58:55.613 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.i.service: Deactivated successfully. 2024-05-17T04:58:55.613 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 systemd[1]: Stopped Ceph mds.i for 0b3ae782-1405-11ef-bc9a-c7b262605968. 2024-05-17T04:58:55.613 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 17 04:58:55 smithi110 systemd[1]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.i.service: Consumed 4min 9.374s CPU time. 2024-05-17T04:58:55.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:58:55.625 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-17T04:58:55.625 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-17T04:58:55.625 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.l 2024-05-17T04:58:55.949 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:58:55 smithi110 systemd[1]: Stopping Ceph mds.l for 0b3ae782-1405-11ef-bc9a-c7b262605968... 2024-05-17T04:58:56.317 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:58:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l[65734]: 2024-05-17T04:58:55.949+0000 7f1d3cbcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-17T04:58:56.318 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:58:55 smithi110 ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l[65734]: 2024-05-17T04:58:55.949+0000 7f1d3cbcb700 -1 mds.l *** got signal Terminated *** 2024-05-17T04:59:00.316 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:59:00 smithi110 podman[74853]: 2024-05-17 04:59:00.047747603 +0000 UTC m=+4.202399698 container died fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-17T04:59:00.798 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:59:00 smithi110 podman[74853]: 2024-05-17 04:59:00.466117598 +0000 UTC m=+4.620769693 container cleanup fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-17T04:59:00.799 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:59:00 smithi110 bash[74853]: ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l 2024-05-17T04:59:00.799 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 17 04:59:00 smithi110 podman[74866]: 2024-05-17 04:59:00.608964176 +0000 UTC m=+0.562630246 container remove fe04b9cc6b4dc023faf9344888a7b8cb2e7457346b9b39b4d525299c06802ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:df105da8a437d75574eedcd495e98a400635440b68e256d61c01d2b33576d60b, name=ceph-0b3ae782-1405-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.012458-debug-931809a, maintainer=Guillaume Abrioux ) 2024-05-17T04:59:00.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0b3ae782-1405-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-17T04:59:00.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-17T04:59:00.850 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-17T04:59:00.851 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T04:59:01.053 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:59:03.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T04:59:03.919 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:59:06.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-17T04:59:06.833 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T04:59:09.272 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:59:09.307 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:59:09.340 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-17T04:59:09.366 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-17T04:59:09.369 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi050/crash 2024-05-17T04:59:09.370 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T04:59:09.405 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T04:59:09.405 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-05-17T04:59:09.407 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi105/crash 2024-05-17T04:59:09.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T04:59:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T04:59:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-05-17T04:59:09.454 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi110/crash 2024-05-17T04:59:09.455 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash -- . 2024-05-17T04:59:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-17T04:59:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-05-17T04:59:09.510 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-17T04:59:09.511 DEBUG:teuthology.orchestra.run.smithi050:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-17T04:59:09.625 INFO:tasks.cephadm:Compressing logs... 2024-05-17T04:59:09.625 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T04:59:09.628 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T04:59:09.631 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T04:59:09.659 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-17T04:59:09.660 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-17T04:59:09.660 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T04:59:09.660 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-17T04:59:09.660 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T04:59:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T04:59:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-17T04:59:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T04:59:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.x.log/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.3.log: 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T04:59:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr: 87.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:gzip/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.d.log: -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-17T04:59:09.672 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-17T04:59:09.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-17T04:59:09.765 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 83.2% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T04:59:09.779 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 83.3% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T04:59:09.789 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.7% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-17T04:59:09.798 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-17T04:59:09.814 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T04:59:09.815 INFO:teuthology.orchestra.run.smithi050.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-17T04:59:09.821 INFO:teuthology.orchestra.run.smithi110.stderr: 90.5% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T04:59:09.824 INFO:teuthology.orchestra.run.smithi105.stderr: 94.6% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-17T04:59:09.833 INFO:teuthology.orchestra.run.smithi105.stderr: 90.5% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T04:59:09.854 INFO:teuthology.orchestra.run.smithi110.stderr: 93.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-17T04:59:09.856 INFO:teuthology.orchestra.run.smithi050.stderr: 83.3% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-17T04:59:09.868 INFO:teuthology.orchestra.run.smithi105.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T04:59:09.923 INFO:teuthology.orchestra.run.smithi050.stderr: 90.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-17T04:59:09.975 INFO:teuthology.orchestra.run.smithi110.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T04:59:10.033 INFO:teuthology.orchestra.run.smithi110.stderr: 94.7% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-17T04:59:10.034 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T04:59:10.072 INFO:teuthology.orchestra.run.smithi110.stderr: 87.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T04:59:10.074 INFO:teuthology.orchestra.run.smithi050.stderr: 90.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-17T04:59:10.168 INFO:teuthology.orchestra.run.smithi050.stderr: 94.7% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-17T04:59:10.171 INFO:teuthology.orchestra.run.smithi050.stderr: 94.7% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-17T04:59:10.174 INFO:teuthology.orchestra.run.smithi050.stderr: 94.7% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-17T04:59:10.229 INFO:teuthology.orchestra.run.smithi110.stderr: 91.1% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-17T04:59:10.531 INFO:teuthology.orchestra.run.smithi050.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-17T04:59:10.544 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-17T04:59:14.459 INFO:teuthology.orchestra.run.smithi105.stderr: 91.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-17T04:59:14.534 INFO:teuthology.orchestra.run.smithi105.stderr: 94.2% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-17T04:59:16.231 INFO:teuthology.orchestra.run.smithi110.stderr: 95.2% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-17T04:59:18.274 INFO:teuthology.orchestra.run.smithi110.stderr: 92.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-17T04:59:24.767 INFO:teuthology.orchestra.run.smithi050.stderr: 90.1% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-17T04:59:28.276 INFO:teuthology.orchestra.run.smithi105.stderr: 97.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-17T05:00:54.478 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-17T05:00:57.858 INFO:teuthology.orchestra.run.smithi105.stderr: 94.6% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-17T05:02:48.568 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-17T05:02:52.307 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-17T05:03:23.213 INFO:teuthology.orchestra.run.smithi050.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-17T05:03:34.827 INFO:teuthology.orchestra.run.smithi105.stderr: 90.8% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-17T05:03:34.830 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-05-17T05:03:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:real 4m25.185s 2024-05-17T05:03:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:user 5m7.118s 2024-05-17T05:03:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m11.884s 2024-05-17T05:05:21.052 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-17T05:06:00.093 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-17T05:06:15.650 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-17T05:06:25.492 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-17T05:06:25.494 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-17T05:06:25.494 INFO:teuthology.orchestra.run.smithi050.stderr:real 7m15.851s 2024-05-17T05:06:25.494 INFO:teuthology.orchestra.run.smithi050.stderr:user 5m56.085s 2024-05-17T05:06:25.494 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m13.583s 2024-05-17T05:09:54.326 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-17T05:09:56.693 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-17T05:10:00.238 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-17T05:11:07.181 INFO:teuthology.orchestra.run.smithi110.stderr: 94.1% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-17T05:12:51.078 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-17T05:13:57.072 INFO:teuthology.orchestra.run.smithi110.stderr: 94.0% -- replaced with /var/log/ceph/0b3ae782-1405-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-17T05:13:57.074 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-05-17T05:13:57.074 INFO:teuthology.orchestra.run.smithi110.stderr:real 14m47.427s 2024-05-17T05:13:57.074 INFO:teuthology.orchestra.run.smithi110.stderr:user 7m0.175s 2024-05-17T05:13:57.074 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m18.941s 2024-05-17T05:13:57.075 INFO:tasks.cephadm:Archiving logs... 2024-05-17T05:13:57.075 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi050/log 2024-05-17T05:13:57.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T05:14:54.316 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi105/log 2024-05-17T05:14:54.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T05:15:45.726 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi110/log 2024-05-17T05:15:45.727 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T05:16:55.983 INFO:tasks.cephadm:Removing cluster... 2024-05-17T05:16:55.984 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force 2024-05-17T05:16:56.207 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T05:16:56.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force 2024-05-17T05:16:57.052 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T05:16:57.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0b3ae782-1405-11ef-bc9a-c7b262605968 --force 2024-05-17T05:16:57.917 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: 0b3ae782-1405-11ef-bc9a-c7b262605968 2024-05-17T05:16:58.684 INFO:tasks.cephadm:Removing cephadm ... 2024-05-17T05:16:58.684 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T05:16:58.705 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T05:16:58.724 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-17T05:16:58.739 INFO:tasks.cephadm:Teardown complete 2024-05-17T05:16:58.740 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-17T05:16:58.751 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-17T05:16:58.751 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T05:16:58.754 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T05:16:58.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T05:16:58.888 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-17T05:16:58.898 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-17T05:16:58.899 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T05:16:58.902 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T05:16:58.904 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T05:16:58.917 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-17T05:16:58.918 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-05-17T05:16:58.919 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-05-17T05:16:58.940 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T05:16:58.940 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-05-17T05:16:58.941 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +858us[ +851us] +/- 62ms 2024-05-17T05:16:58.941 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -722us[ -729us] +/- 61ms 2024-05-17T05:16:58.941 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -112us[ -112us] +/- 74ms 2024-05-17T05:16:58.941 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +876us[ +875us] +/- 62ms 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -730us[ -730us] +/- 61ms 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -95us[ -96us] +/- 74ms 2024-05-17T05:16:58.942 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T05:16:58.948 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-17T05:16:58.949 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-17T05:16:58.949 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +849us[ +853us] +/- 62ms 2024-05-17T05:16:58.949 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -724us[ -724us] +/- 61ms 2024-05-17T05:16:58.949 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -110us[ -110us] +/- 74ms 2024-05-17T05:16:58.949 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-17T05:16:58.950 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-17T05:16:58.959 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-17T05:16:58.960 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-17T05:16:58.991 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T05:16:59.128 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T05:16:59.257 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-17T05:16:59.368 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T05:16:59.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:16:59.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T05:16:59.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:16:59.432 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-17T05:16:59.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:16:59.461 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-17T05:16:59.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-17T05:16:59.500 INFO:teuthology.task.internal:Duration was 3658.131217 seconds 2024-05-17T05:16:59.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-17T05:16:59.582 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-17T05:16:59.583 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T05:16:59.587 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T05:16:59.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T05:16:59.645 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T05:16:59.651 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T05:16:59.652 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-17T05:17:00.117 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-17T05:17:00.118 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-05-17T05:17:00.118 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T05:17:00.143 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-05-17T05:17:00.144 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T05:17:00.177 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-05-17T05:17:00.178 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T05:17:00.204 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-17T05:17:00.205 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T05:17:00.208 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T05:17:00.221 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T05:17:00.272 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-17T05:17:00.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T05:17:00.279 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T05:17:00.292 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T05:17:00.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-17T05:17:00.858 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-17T05:17:00.858 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T05:17:00.884 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T05:17:00.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T05:17:00.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-17T05:17:00.961 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:00.963 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:00.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:00.999 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-05-17T05:17:01.000 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-05-17T05:17:01.022 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-05-17T05:17:01.040 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:01.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:17:01.076 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:01.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:17:01.092 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T05:17:01.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T05:17:01.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-17T05:17:01.123 INFO:teuthology.task.internal:Transferring archived files... 2024-05-17T05:17:01.124 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi050 2024-05-17T05:17:01.125 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T05:17:01.181 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi105 2024-05-17T05:17:01.182 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T05:17:01.219 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_03:38:57-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7710343/remote/smithi110 2024-05-17T05:17:01.219 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T05:17:01.256 INFO:teuthology.task.internal:Removing archive directory... 2024-05-17T05:17:01.256 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T05:17:01.259 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T05:17:01.261 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T05:17:01.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-17T05:17:01.319 INFO:teuthology.task.internal:Not uploading archives. 2024-05-17T05:17:01.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-17T05:17:01.350 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-17T05:17:01.350 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T05:17:01.353 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T05:17:01.355 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T05:17:01.366 INFO:teuthology.orchestra.run.smithi050.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 05:17 /home/ubuntu/cephtest 2024-05-17T05:17:01.368 INFO:teuthology.orchestra.run.smithi105.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 05:17 /home/ubuntu/cephtest 2024-05-17T05:17:01.370 INFO:teuthology.orchestra.run.smithi110.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 05:17 /home/ubuntu/cephtest 2024-05-17T05:17:01.371 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-17T05:17:01.399 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-17T05:17:01.517 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 3658.13121676445 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-17T05:17:01.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T05:17:01.607 INFO:teuthology.run:pass